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Preface 



For the twenty-seventh year, the Research and Theory Division of the Association for Educational 
Communications and Technology (AECT) is sponsoring the publication of these Proceedings . This is 
Volume #2 of the 27 th Annual Proceedings of Selected Papers On the Practice of Educational 
Communications and Technology Presented at The National Convention of the Association for Educational 
Communications and Technology held in Chicago, IL Copies of both volumes were distributed to 
Convention attendees on compact disk Volume #2 will also be available on microfiche through the 
Educational Resources Clearinghouse (ERIC) system. 

This volume contains papers primarily dealing with instruction and training issues. Papers dealing with 
research and development are contained in the companion volume (27 th Annual, Volume #1), which also 
contains over 100 papers. 

REFEREEING PROCESS: Papers selected for presentation at the AECT Convention and included in these 
Proceedings were subjected to a reviewing process. All references to authorship were removed from 
proposals before they were submitted to referees for review. Approximately sixty percent of the 
manuscripts submitted for consideration were selected for presentation at the convention and for 
publication in these Proceedings . The papers contained in this document represent some of the most current 
thinking in educational communications and technology. 

M. R. Simonson 
Editor 
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Modular Object-Oriented Dynamic Learning Environment: 
What Open Source Has To Offer 



Pavlo Antonenko 
Serkan Toy 
Dale Niederhauser 
Iowa State University 

Open source online learning environments have emerged and developed over the past 10 
years. In this paper we will analyze the underlying philosophy and features of MOODLE 
based on the theoretical framework developed by Hannafin and Land (2000). 
Psychological, pedagogical, technological, cultural, and pragmatic foundations comprise 
the framework and represent the major points of our analysis. This paper is intended for 
instructional designers, distance education specialists, K-12 and college instructors who 
may want to add an online component to their courses. 

As we enjoy great Advantages from the Inventions of others we 
should be glad of an Opportunity to serve others by any Invention of 
ours, and this we should do freely and generously. 

~ Benjamin Franklin 



Introduction 

Benjamin Franklin’s philosophy resonates in the recent advent of open source software. The term “open 
source” refers to computer programs or operating systems for which the source code is publicly available. (Johnson- 
Eilola, 2002) The definition further explains that inherent in the open source philosophy is the freedom of a 
distributed community of programmers to modify and improve the code (Perens, 1999). 

Open source software promotes the use of technologically -neutral, non-proprietary tools and formats, 
which allow for wide-spread access. According to the Open Source Initiative website (2004), the major reasons for 
utilizing open source software include free distribution, freedom to modify the software to meet individual needs, 
cross -platform compatibility and universal accessibility, and active collaboration to improve design. These factors 
bear a special significance in an educational setting. As Terry Vessels (2004) puts it: 

...Educators have been called upon throughout history to combat censorship imposed by various 
powers over the flow of information. The censorship being applied today comes in the form of 
licenses that lock away the tools to build the information age and laws that limit fair use in ways 
that are unprecedented in the modern era (f 2). 

The open source movement has already had a significant impact in the business world (Wheeler, 2003), and 
is now drawing the attention of educators around the globe. Distance education is at the forefront of using and 
creating open source applications in education. 

Current advances in open source online learning environments are a response to the shortcomings of 
commercial products like WebCT and Blackboard. One such weakness is a lack of flexibility in designing and 
adding customized learning modules. With commercial products one can only include elements that the software 
designers deemed necessary when they developed the program. With an open source learning environment it is 
possible to download and use any learning module one might find on any open source software website. This opens 
almost limitless capabilities for the user to customize the application by choosing from a variety of options for e- 
rnail, discussion boards, chat, online quizzes, and all the other elements one might want to include. 

Further, as the open source definition suggests, the actual code can be modified and improved to meet 
individual needs. So, if the user decides that an open source module he or she found is almost perfect, the code can 
be modified to meet his or her needs. Look, feel and functionality can all be changed since the code can be easily 
accessed and modified. 

As to pricing of commercial products, “companies are moving toward selling campuswide access to 
software, and toward setting prices based on the number of students each college enrolls.” (Young, 2002) According 
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to Young, the company's [WebCT] current software costs $3,000to $30,000 annually, depending on the size of 
the institution and the level of use of the software.” This is particularly important with seemingly continual 
decreases in federal and state appropriations for higher education. All open source software, on the other hand, is 
available free of charge to anyone who wants to use it. 

Shortcomings of commercial distance education software have prompted the development of a number of 
open source online learning environments such as MOODLE, EduCreate, Covidia, and LogiCampus. Concurrently, 
the open source concept has developed to the point that even the tools used to create such systems are open source. 
For example, most of this software is written in Hypertext Preprocessor (PHP) an open source alternative to 
commercial scripting languages and make use of open source relational database systems like MySQL. They can be 
installed on almost any web server — the most popular being Apache (again, open source). One open source online 
learning environment. Modular Object-Oriented Dynamic Learning Environment (MOODLE) is a highly usable, 
reliable, and functional alternative to popular commercial products like WebCT and Blackboard. Developed by 
Martin Dougiamas, a PhD student in Computer Science and Education at Curtin University of Technology, 
Australia, this learning environment provides the considerable flexibility inherent in open source software for 
designing and administering Web-based courses to meet the individual needs of online educators. It was specifically 
developed to address the aforementioned shortcomings of commercial online learning environments — which he had 
used and supported in his own teaching and as a technician working with faculty. In his own words: 

... It started in the 90's when I was a webmaster at Curtin University of Technology and a system 
administrator of their WebCT installation. I encountered many frustrations with the WebCT beast 
and developed an itch that needed scratching - there had to be a better way (no, not Blackboard :) 
(Dougiamas, 2004, f 2). 



Analysis of MOODLE 

Dougiamas decided to create his own online learning environment in the open source format and allow the 
open source community to help develop and refine his ideas. MOODLE was designed to support and promote users 
interested in developing constructivist, student-centered learning environments (Dougiamas, 2004). To examine this 
claim, we conducted an analysis of MOODLE using a framework developed by Land and Hannafin (2000) which 
was initially designed as a guide for developing constructivist learning environments. According to the authors, 
‘‘Learning environments, directed as well as constructivist, are rooted in five core foundations: psychological, 
pedagogical, technological, cultural, and pragmatic” (Hannafin & Land, 1997). Figure 1 highlights the five 
components of these core foundations as applied to the design of student-centered learning environments. 
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| Pedagogical 

Authenticity 
Anchoring 
Learner centered 
Problem based 
Scaffolding/ guiding 
Manipulating 
objects/ ideas 
Constructing 
artifacts 

Mcaningfulncss of 
errors 



Psychological 

• Situated cognition 

• Dominance of prior 

knowledge and 
experience 

• Role of context in 
learning and 
applying knowledge 

• Personal theories, 

models, & beliefs 



Cultural 

• Inquiry and discovery 

• Depth versus breadth 

• Critical thinking 

• Evaluation and 

synthesis 

• Skill accountability 

less important than 
problem generation 
and reasoning 



Technological 

• Tools 

- build, test beliefs 

- visualize 

- cognitive tools 

- communication 

• Resources 

* databases 

- sound clips 
video libraries 

- encyclopedia 




Pragmatic 

• Usable in target 

settings 

• Aligned with key 

teaching/ learning 
goals 

• Accountability needs 

addressed 

• Appropriate teacher 

and learner support 
provided 



Figure 1: Five core foundations of student-centered learning environments (Hannafin & Land, 1997) 



In the remainder of the paper we will relate each of Hannafin and Land’s core foundations to the 
underlying philosophy and principles that guided the development of MOODLE. 

Psychological Foundations 

Psychological foundations address research, theory and practice associated with how people think and 
learn. Our examination of the program shell and standard MOODLE learning modules reflect the designers’ use of 
several important considerations from cognitive psychology: 

Situated cognition suggests effective learning should involve immersing students in authentic activity and 
culture in a real-world learning context. (Brown, Collins, & Duguid, 1989). Relevance is enhanced through 
interconnected, embedded engagement with interesting and complex tasks situated in an authentic context. Situated 
learning integrates four critical factors that maximize student learning potential: content, context, community, and 
participation (Stein, 1998). MOODLE learning modules allow instructors to set up complex, ill-defined and 
authentic tasks in real-life contexts, and assign roles for students to assume in the solution of these problems. For 
example, the Workshop module allows students to collaborate on the design of possible solutions to an authentic 
problem and peer assess the suggested solutions. The instructor might decide to have students work individually or 
in groups to determine where to build an alternate route to help alleviate traffic accidents at a specific intersection. 

In the Resource module the instructor can scaffold the activity by providing area maps, different perspectives on the 
problem (e.g. interviews with Department of Transportation experts), and sample solutions devised by other states or 
counties. Students can brainstorm possible solutions in Chat or Discussion forums, then pre sent their alternate routes 
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via the Workshop module, which allows for peer assessment. Thus, learners will compare and contrast their 
solutions and select the best one based on the group discussions. 

Cognitive flexibility is defined as “the ability to adaptively re -assemble diverse elements of knowledge to 
fit the particular needs of a given understanding or problem-solving situation” (Spiro & Jehng, 1990). This theory 
focuses on learning in complex and ill-structured domains — which represent many real-life situations. “A central 
claim of cognitive theory is that revisiting the same material, at different times, in rearranged contexts, for different 
purposes, and from different conceptual perspectives is essential for attaining the goals of advanced knowledge 
acquisition” (Spiro, Feltovich, Jacobson, & Coulson, 1991). One of the major metaphors employed by the theory is 
that of a “... criss-crossed landscape, with its suggestion of a non-linear and multi-dimensional traversal of complex 
subject matter, returning to the same place in the conceptual landscape on different occasions, coming from different 
directions” (Spiro, Vispoel, Schmitz, Samarapungavan, & Boerger 1987; Wittgenstein, 1953). MOODLE supports 
the “criss-crossing of the conceptual landscape” through the Glossary learning module-a unique feature that allows 
users (both students and instructors) to create an online hypertext -based dictionary that is created on the fly and 
updates automatically to all content in the course or even throughout the entire portal. This module allows users to 
consult the glossary dynamically while navigating through the lesson content, assignments, or even discussion 
postings, but, perhaps more importantly, students take ownership for their learning as they actively construct a 
richer, more complex and sophisticated learning environment. 

Pedagogical Foundations 

Pedagogical foundations include the instructional practices that the designers use. They are grounded in 
theories of learning and reflect the teaching strategies with which they are aligned. MOODLE developers have 
explicitly stated that the design of the software is grounded in constructivist and constructionist instructional 
principles. In the following paragraph, Dougiamas (2004) discusses the importance of pedagogy and encourages 
educators to adopt the constructivist methodology: 

Once you are thinking about all these [pedagogical] issues, it helps you to focus 
on the experiences that would be best for learning from the learner's point of 
view, rather than just publishing and assessing the information you think they 
need to know. It can also help you realise how each participant in a course can 
be a teacher as well as a learner. Your job as a 'teacher' can change from being 
'the source of knowledge' to being an influencer and role model of class culture, 
connecting with students in a personal way that addresses their own learning 
needs, and moderating discussions and activities in a way that collectively leads 
students towards the learning goals of the class. ( ( |[ 13) 

Constructivism implies that the learner links new information with existing and future -oriented knowledge 
in unique and meaningful ways (McCombs & Whisler, 1997). Social constructivism, a branch of this theory, 
emphasizes the value of knowledge that is built socially in a learning community. Pioneered by theorists like 
Vygotsky (1978), this paradigm argues for the importance of culture and context in forming understanding. Learning 
is not a purely internal process, nor is it a passive shaping of behaviors . Vygotsky favored a concept of learning as a 
social construct which is mediated by language via social discourse. 

MOODLE promotes social discourse in learning through the synchronous and asynchronous 
communication modules described above. Internal support for introducing groups within a class of students in the 
learning environment is built into the program. Students can form cohorts themselves or the instructor can moderate 
this process. Within a cohort students work cooperatively and engage in a more individualized interaction with one 
another. Later, cohorts can share their perspectives in a whole class discussion and continue the learning process as a 
unified group. Although the program supports this type of instruction, the art of combining individual activities with 
cohort -based or whole -class activities is one of the factors that reflect the teaching skills of the instructor. An 
instructor may choose to integrate a discussion forum, chat or even a private two-way dialogue into any learning 
activity of the course. Wiki is the module that gives students and instructors the opportunity to collaborate on the 
design of hypertext that represents the knowledge constructed socially by the learning community of an individual 
class. A community of graduate students, for example, can thus work together on a literature review for a specific 
topic. The document is started when a student makes the first text entry. Other students modify this document to 
develop the ideas for the literature review. The system keeps track of each modifications and both students and 
instructor have the opportunity to see how the document developed over time, and who and how each person 
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contributed. Younger learners might enter and revise class notes in a group Wiki, compare and revise the notes as a 
class, and create a useful resource for each other and for future students. 

Constructionism (as opposed to Constructivism) asserts that that knowledge acquisition is particularly 
effective when constructing something for others to experience. Papert (1991), who started developing this concept 
in the 1980s, asserts that constructionism “.. .shares constructivism’s connotation of learning as "building knowledge 
structures" irrespective of the circumstances of the learning. It then adds the idea that this happens especially 
felicitously in a context where the learner is consciously engaged in constructing a public entity, whether it's a sand 
castle on the beach or a theory of the universe.” Constructionism stresses the importance of building external 
artifacts as a means to more effectively construct and represent the inner knowledge structures. The importance of 
learning through design is supported by the research on children’s development of strategies and collaboration in 
video game design, building and learning with programmable bricks (Kafai & Resnick, 1996). In addition to the 
Glossary and Wiki examples provided earlier, MOODLE supports the construction of artifacts by allowing learners 
to present and share their knowledge in a variety of diffe rent formats, including multimedia and hypertext. These 
products can then be shared with others through Discussion Forum , Resource or Assignment modules. Further, as 
described above, the Workshop module promotes social construction of knowledge artifacts by allowing students to 
collaborate on the possible solutions to ill -structured real-world problems and evaluate them in the peer assessment 
activity. 

Technological Foundations 

Technological foundations affect how media can support, limit, or improve the pedagogy of the learning 
environment. According to Land and Hannafin (2000), “... technology can control the pacing and chunking of 
information where cognitive load limitations are assumed...” 

MOODLE supports the pedagogy of the learning environment through its interactive, collaborative and 
reflective modules. For example, the Journal module allows the instructor to ask the students to reflect on a 
particular topic, and edit and refine their answer over time. This activity promotes self-assessment, critical thinking, 
and metacogntition. Learning journals entice students to think in unconventional ways (Fulwiler 1987) and provide 
an opportunity to both develop and capture reflection in the learning process (Moon, 1999). 

Although the flexibility of hypertext systems is powerful, it may result in disorientation problems (Theng, 
1997) and information overload (Niederhauser, Reynolds, Salmen & Skolmoski, 2000). The MOODLE shell 
provides a framework for presenting modules that accounts for these potential problems . It allows to structure and 
control the presentation of the learning material and decrease the risk of the ‘‘lost in hyperspace” problem (Boyle & 
Encarnacion, 1994). For example, each page of the portal has a quick-jump drop-down menu that allows users to 
navigate the system more efficiently. Users may also use the personalized navigation bar at the top of each page, 
which tracks and shows the history of previously viewed pages for each specific user. Extraneous cognitive load 
often occurs when instructional materials require learners to use cognitive resources to search for specific 
information without providing any scaffolds or quick and easy access to relevant resources (van Merrienboer, 
Kirschner, Kester, 2003). Information overload of learners is reduced through built-in support for adaptive hypertext 
navigation (Hook, 1997). Navigation in MOODLE is further enhanced via the use of the Latest News section, which 
allows instant access to the most recent discussion postings, news entries and assignments. Asimilar function is 
performed by the Calendar module that highlights the upcoming events, due dates and other information through 
simple mouse rollovers. 

A major strength of using MOODLE lies in the inherent nature of open source software which promotes 
customization. With only a basic knowledge of web scripting one can add almost any open source stand-alone 
application to customize an online learning environment to meet individual needs. For example, more than half of 
the modules, visual themes, and administration features available in the current version of the program have been 
added by different members of the open source community. This brings us to the discussion of another important 
foundation of learning environments - the cultural aspect of design. 

Cultural Foundations 

Cultural foundations tend to reflect the prevailing values of a learning community. For instance, one might 
find particular values such as back to basics, interdisciplinary learning, or global society in a given learning 
environment. (Land & Hannafin, 2000) 

The central ideas in the MOODLE culture are collaboration, sharing and community. They are represented 
in Dougiamas’ (2004) discussion of social constructivism, one of the four major concepts in the underlying 
philosophy of MOODLE: 
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[social constructivism] extends the above ideas into a social group constructing things for one 
another, collaboratively creating a small culture of shared artifacts with shared meanings. When 
one is immersed within a culture like this, one is learning all the time about how to be a part of 
that culture, on many levels, (f 9) 

Virtual community is defined as a community of people sharing common interests, ideas, and feelings over 
the Internet or other collaborative networks (Rheingold, 2000). Members of a learning community also share the 
meanings that they make of the learning material. Social exchanges by individual students are an important part of 
the group interaction and learning. They help build a sense of trust and respect among community members (Lally & 
Barrett, 1999). Students in the MOODLE learning environment form a cultural community by interacting in 
synchronous and asynchronous discussion modules. Journal and the collaboration tools like Workshop , Wiki and 
Glossary discussed earlier. 

Another aspect of community building associated with MOODLE involves those who are working to 
develop, refine, and support advancement of the program. The collaborative nature of designing and supporting 
open source applications like MOODLE reflects the inportant social aspect of software development and 
knowledge construction by integrating the diverse perspectives and expertise of the members of the international 
community working together to improve the quality of the software. The community of MOODLE developers and 
users bring multiple perspectives and skills and share their views on online learning in MOODLE discussion forums. 
This international collaborative effort results in a truly socially constructed design process, which enhances the 
quality of the software from both the pedagogical and technological perspective. For example, Williams Castillo, “a 
curious developer” from Caracas, Venezuela, contributed to this open source project by designing the Glossary 
module. He also maintains a discussion forum on creative uses of the Glossary. Further, the community works 
collaboratively to provide technical support for all of its members. 

Pragmatic Foundations 

As the name suggests, these foundations are concerned with doing a reality check. How does the learning 
environment correspond to the needs of target audiences? What are the benefits, and what are the limitations? 

MOODLE is an open source online learning environment that is developed for the administrators of web- 
based courses for K-12 and university instructors. The system is efficient and features cross -platform compatibility 
and a low-tech browser interface. A highly relevant aspect of MOODLE for educators is that it is available as a free 
download on the Internet and can be installed in an hour. Even though one can administer the installation of 
MOODLE with just a basic knowledge of web interfaces, one should probably have experience managing database- 
driven dynamic websites. 

With the software installed the creation and management of the learning environment can be performed by 
a person with limited technological expertise. MOODLE’s creators realize that educators are not high-end 
developers and therefore all the administration is performed through a simple, intuitive graphical user interface 
(Figure 2). Help buttons are included for every component of the administration menu and provide guidance for 
novice users of the system. 

Most text entry areas in MOODLE such as resources, forum postings, assignments etc. can be edited using 
an embedded WYSIWYG (What You See Is What You Get) HTML editor. The administrator (instructor) can allow 
or prohibit students to modify specific parts of the course, like journal entries. Further, MOODLE courses can be 
categorized and searched, allowing one MOODLE installation to support thousands of courses and function as a 
campus edition. Plug-in activity modules can be added to existing MOODLE installations and enhance the existing 
structure of the courses. Customizable themes allow the administrator to customize the site colors, fonts, layout, and 
other features to suit individual needs. MOODLE language packs allow full localization to 43 languages. Even the 
language packs can be easily edited using a built-in web-based editor. [Insert Figure 2 about here] 

Technical support is freely available on the Web and is provided by MOODLE developers and users 
through discussion forums and Frequently Asked Questions section. Each learning module is supported by 
a separate discussion forum containing tips and tricks, teaching strategies, learning standards, course 
formats, and advice on how to build a strong learning community. 

Conclusion 

Open source software has become increasingly popular in many areas. One such application, MOODLE, 
provides a constructivist learning environment that makes a significant contribution to enhance web-based learning. 

As this paper demonstrates, the design of MOODLE integrates general principles of constructivist learning 
and provides an online learning context that supports student-centered pedagogy. This system is grounded in 
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situated cognition and cognitive flexibility theory that provide the opportunity for an instructor to create a 
constructivist and constructionist environment to enhance teaching and learning. The capabilities to design student- 
centered learning are embedded in MOODLE and its modules; however, it is ultimately the responsibility of the 
instructor to make good use of them. 

The MOODLE project indicates the growing interest of educators and open source programmers in joining 
their efforts to improve the quality and reduce the cost of education. Since it is distributed under the General Public 
License, MOODLE can be easily modified to meet individual needs. Further, development, customization, and 
support are all completed as part of the community effort to improve online teaching. This open source application 
provides an effective and cost-efficient alternative to expensive commercial software packages for those interested 
in joining the movement to provide high quality constructivist-based educational experiences in the online 
environment. 
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Jazz It Up, with Music Technology! 
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As the song goes , “Don’t know much about His -to-ry, don’t know much about Tech-no-lo-gy !” But what 
we do know is that music technology now plays a crucial role in most schools. Fully integrating this technology 
requires much greater awareness. From Mozart to Madonna, technology has forever changed the field of music. 
Although accessing music through the Web and through digital storage devices has been remarkably significant, 
perhaps to an even greater extent, music synthesizers and editing software have dramatically changed the very 
nature of music. Technology provides powerful aids to composing, notating, editing, and performing music that 
even elementary school students can learn to use. 

Cakewalk Sonar (Image 1) represents affordable state-of-the-art digital recording software that compares to 
expensive studio facilities. There's a lot of flexibility built into Sonar to achieve a variety of musical forms including 
recording audio, creating MIDI files, looping and sequencing, and adding effects. Any style of music can be created 
using Sonar's MIDI and audio systems and there are also DX instruments and automated effects. 

Band-in-a-Box (Image 2), described as an intelligent accompaniment software, is a powerful and creative 
music composition tool for exploring and developing musical ideas with near-instantaneous feedback, ft contains 
features to display notation, enter lyrics, create melodies, add harmonization, and program a variety of musical 
styles. The Soloist generates professional quality solos over any chord progression. The Melodist creates songs from 
scratch with chords, melodies, intros, solos, and even a title. 

Band-in-a-Box can add recordings of acoustic instruments or voices to the composition with special effects 
processing. Its built-in audio harmonies can turn an audio track into multiple harmony parts and even adjust its pitch 
by tracking the Band-in-a-Box melody track. Digital audio features make Band-in-a-Box the perfect tool for 
creating, playing, and recording music with MIDI, vocals, and acoustic instruments. You can print out your musical 
arrangement with repeats and endings, DC markings and codas, or save it as a graphic file for Web publication or 
even e-mail it to a friend. And when you're ready to let others hear your composition, you can burn it directly to an 
audio CD. Save your composition as a Windows Media File or other compressed formats for a file that's "Internet 
ready." It’s a great way to create your won backup band! 

PowerTracks Pro Audio (Image 3) provides a fully -featured MIDI and digital audio sequencer and multi- 
track recorder. Unlike Band-in-a-Box, you do not create music in PowerTracks by typing in the chords to a song. 
Rather, you 'layer' tracks of MIDI and digital audio, each of which must be recorded from scratch. This takes longer, 
but in exchange you have much more control over the nuances of your music. Band-in-a-Box does not allow you to 
edit the individual events of the accompaniment tracks. This is because the program generates the accompaniment 
for you, and it is different every time you play your song. PowerTracks also comes loaded with a host of effects to 
help you put the subtle, finishing touches on your work. Use a bit of reverb to create a 'spacious' feel; add some 
chorus and distortion to enhance the 'grind' of your guitar tracks; fiddle with the compressor to give your drums that 
extra 'punch.' PowerTracks (and patience) are all you need to infuse your songs with a refined, 'studio' feel. 

If you've ever had an original song idea in your head and wished you could have it performed, MIDI is the 
way to do it. All you need is a MIDI Sequencer, plus a MIDI instrument to enter notes. You can also use MIDI 
Notation software to place notes on a musical staff without playing them at all. You can start with just a melody and 
then add backing chords, bass, and rhythm later, or add instruments in any order you like. If you make mistake, you 
can change it without having to play the part all over again. You can also make entire sections repeat without 
playing them again. And you can rearrange and re -orchestrate your song as many times as you like. 

Many people enjoy arranging and orchestrating music as much as performing it. There are MIDI files 
available for songs from every style of music, as well as, software programs that generate the basic rhythm and 
chord patterns that define specific styles that you can use to create your own arrangements and orchestrations. Just 
change the instrumentation, add a verse or chorus here or there, even put in your own original phrase or section. All 
of this is easy to do with MIDI. You can also share your arrangements with other people, who can then rearrange 
them to fit their own needs. Many people download MIDI files from the Internet and arrange them to fit their own 
needs. 

Finale (Image 4) offers the most features of any music software available using convenient templates, 
automated note entry, editing, and printing. To lay the melody down, you simply grab an instrument and play the 
melody into a microphone. Then you apply the Auto -Harmonizing assistant. Seconds later, a beautiful 6 -part 
composition has been created. Using the Drum Groove Plug-in and the percussion section, you add a rhythmic 
background. To complete the arrangement, you employ the Rhythm Section Generator to instantly create piano, bass 
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and drum parts. Next you apply some simple edits to give it your personal touch. Finally, you extract and print the 
parts and burn it to an audio CD, then slap the CD into the portable player and enjoy the results. The newest version 
supports both Macintosh and Windows and features for music educators. 

Imagine creating an instructional file for your students that can be opened in a downloadable shareware 
application called Finale Performance Assessment, or FPA (Image 5). Here they practice the music by adjusting the 
tempo, hearing how it is supposed to be played, and playing it over and over until ready for a performance. The 
student is automatically recorded in a file they can save to disk for their portfolio, or email to the instructor, family, 
or friends. Their performance is assessed on-screen with notes they played correctly appearing in green and incorrect 
notes in red. FPA provides for MIDI keyboard, woodwind and brass players. 

After creating music, playback usually occurs through devices called synthesizers. Although most 
computers have built-in synthesizer capability on their sound cards, more serious performers use specialized 
hardware or software to generate the sounds through their computer. Musical-Instmment-DigitaHnterface (MIDI) 
is a technology that represents music in digital form. Unlike other digital music technologies such as MP3 and CDs, 
MIDI messages contain individual instructions for playing each individual note of each individual instrument. So 
with MIDI it is actually possible to change just one note in a song, or to orchestrate and entire song with entirely 
different instruments. Since each musical part in a MIDI performance is separate from the rest, it’s easy to listen to a 
single instrument and study it for educational purposes, or to mute individual instruments in a song so that you can 
play that part yourself. Hardware synthesizers can be expensive but software alternatives are quite inexpensive or 
even included with most arranging and composition programs . For example, the Edirol Virtual Sound Canvas 
(Image 6) allows you to customize playback of MIDI files using the same sounds and for a fraction of the cost of 
hardware synthesizers. You can even export your finished files into popular multi- media application formats. 

The Superscope PSD300 (Image 7) is the world’s first CD recording system for the performing arts that 
combines a professional quality portable CD recorder with a CD player that is designed specifically as a rehearsal 
and performance tool. The CD player features special controls that allow musicians to practice with their favorite 
artist or accompaniment CD and manipulate the key or tempo in real time. The voice reduction feature reduces 
specific sounds so a performer can sing along or play an instrument with a recording like a Karoke system. While 
not perfect, it certainly is useful. 

From beginner to the professional, technology has forever changed the way we create music. 
Experimentation with music scanning and pattern recognition software may soon make it possible to capture not 
only existing scores and recordings but allow even non-musicians to explore their ideas and produce unique 
arrangements and compositions, all with simple clicks of a mouse ! 
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Image 1: Cakewalk Sonar Audio Recording Software 




Image 2: Band-In -A-Box Auto-accompaniment Software 
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Image 3: PowerTracks Pro Audio Sequencer Software 
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Image 4: Finale Music Composition Software 
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Image 5: Finale Performance Assessment Software 




Image 6: Edirol Virtual Sound Canvas Software Synthesizer 
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Image 7: Superscope PSD300 Dual-Well Portable CD Recorder 
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Motivations for Older Adults to Enroll in Computing Classes 

As the Baby Boomers mature and modem medicine contributes to a rising life expectancy, the American 
Senior citizen population is growing and the demand for more educational opportunities for them is as well. 
Computer education and training, in particular, is becoming more attractive to older adults, in part because: 

• More and more of them are living in the community at large, rather than nursing homes (Mehrotra, 2003), 
yet many find their world shrinking due to reduced mobility (Coughlin, 1999). 

• The media continues to broadcast the importance of engaging the mind in complex tasks and activities to 
promote optimum cognitive functioning (Irizarry, Downing & Elford, 1997; Rowe & Kahn, 1998; Czaja, 
Guerrier, Nair, & Landauer, 1999) 

• The newest generation of older Americans are more educated than past generations, and therefore, more 
interested in continuing to learn and finding out about available learning opportunities (Irizarry, et.al., 

1997; Mehrotra, 2003; Purdie & Boulton-Lewis, 2003), especially those geared just for them (Dunnett, 
1998; Redding, Eisenman, & Rugolo, 1998; Timmerman, 1998; Morrell, Mayhorn, & Bennett, 
2000;Seniors, 2004) 

• Older adults have the desire to reduce feelings of isolation, to connect socially with their peers, and to keep 
a network of companionship and mutual support active (Gallant, 2003; Mehrotra, 2003; Hudson 1996) 

• Many older adults do not want to become obsolete in dealing with societal changes (Mehrotra, 2003), 
including technology (Irizarry, et.al., 1997; Burwell, 2001; Purdie and Boulton-Lewis, 2003) 

• Many older Americans want to develop personal or groups skills so that they can continue to give service 
to others (Mehrotra, 2003) 

• Less information that older adults need is available in print, as it is being migrated by government agencies, 
service organizations and corporations to the Web (Seniors, 2004) 

• As older adults hear from their computer-using peers about what they are using a computer for, they are 
anxious to learn particular tasks that are useful to them (Knowles, Holton, & Swanson, 1998) 

As a result, more and more older adults are hungry for intellectual stimulation and the opportunity to 
expand knowledge, learn new skills, and make new friends through computer education. They seek opportunities 
that are geared just for them and that address their interests. For example. Seniors are particularly attracted to 
sessions on e-mail/communicating with family and friends, finding ancestors, tracking investments, using the 
electronic library for book and magazine-related information, securing information about prescription drugs and 
medical conditions, discovering new interests, accessing newspapers from their birthplace or former communities, 
locating travel and other bargains (e-bay?), shopping for "hard to find" items, checking the weather or news events 
in cities where family and friends live, writing memoirs or a book, managing financial matters, and gaining car-less 
access to religions services, cultural activities, and educational opportunities (Whelen, 1998; Iziarry & Downing, 
1998; Morrell, Mayhorn & Bennett, 2000; Lenhart, Horrigan, Rainie, Allen, Boyce, & O'Grady, 2003; Seniors, 
2004). 

Simply signing up for computer education classes open to the general public is often not the answer, 
however. The needs of older adults are not the same as younger computer-interested learners. Even the older adult 
population is not one-dimensional. "Individual differences in physical, cognitive, and social development tend to 
increase substantially with advancing age. The older the people become, the more dissimilar they become” 
(Mehrotra, 2003, p. 647). Whole groups of older adults, for example, do not fit the profiles described above, such as 
diverse populations who may be less literate, less comfortable with the English language, or more isolated from 
affordable educational opportunities. Although these Seniors may not have ready access to educational opportunities 
as much as their more educated and urban peers, they need education just as much for many of the same reasons. 
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Consequently, the challenge of designing effective educational opportunities for this diverse group of Seniors is 
worth examining. As the Senior population continues to grow, most educational providers in the public, private, and 
non-profit sector will be working to target a growing proportion of their educational programming toward the 
special needs of this age group. 

Barriers for Older Adults to Enroll in Computing Classes 

It is critical that computer education providers keep in mind the many potential barriers that can affect 
whether older adults take advantage of computer-education opportunities or whether they find the experience a 
valuable one. Finances to purchase a computer or Internet access, for example, may be insufficient (Morrell, et.al., 
1997; Mandl, Katz and Kohane, 1998; Seniors, 2004). Geographic distance and available transportation to the class 
provider may be time or cost prohibitive (Seniors, 2004). Disabilities due to aging, such as hearing, vision, stiff 
joints and the need to learn at a sbwer pace may prevent them from fully functioning in classes that do not take 
these conditions into account (Seniors, 2004). Utilization of confusing computer terms without adequate explanation 
can frustrate them (e.g., icon, font, graphic, and the difference between find, search, and submit). Even perceptions 
of some older adults that learning to use a computer is complicated and difficult may keep some of them away 
(Williamson, 1995). But most of all, Seniors want classes that are designed especially for them, i.e., acknowledging 
the foibles of aging, the particular interests they have, and their level of computer skills (Seniors, 2004). 

Education Needs of Older Adults and Complementary 
Recommendations for Computing Classes 

Classrooms/labs for senior learners need to be flexible enough to meet their physical needs. Each Senior 
should be asked what individual needs they might have before as well as during class. Some may not realize that 
they will have problems until they are in the classroom. Determining specific disabilities ahead of time can assist the 
planning team in securing appropriate adaptive devices to facilitate participation and learning (Williamson, Jenkins, 
Wright, Stillman, & Schauder, 2001). Some common accommodations that may be needed include (Hudson, 1996; 
Seniors, 2004; Stevens, 2003): 

• larger size print on the blackboard and in print handouts; effective use of larger print, bold headings, 
pictures/screen captures, white space, and diagrams support independent learning, review and practice 

• larger fonts and an ability to adjust the contrast on the computer monitors 

• wheelchair access to the building 

• no stairs into the building or to the classroom, or access to an elevator 

• adjustable tables with cords off the floor and plenty of leg room to accommodate stiff limbs or a wheelchair 

• good lighting 

• a wireless microphone that enhances the instructor’s clear, slow, and adequately loud voice 

• entrances and restrooms as near to the classroom as possible 

• a shorter class time to reduce fatigue (Ross-Gordon, 2003; Morrow, 2002) 

• accessibility features in Microsoft Windows (if available) 

• when appropriate, assistive devices such as touch screens, switches, larger keyboards, screen readers, or 
trackballs (Williamson et. ah, 2001) 

Malcolm Knowles describes the adult learner as “self-directing” (Knowles, Holton, & Swanson, 1998). 
Typically, Seniors come to class with specific goals, expectations and learning objectives for the time, energy and 
money they have invested (Hadfield, 2003). They are particularly cautious at the start of a learning opportunity 
because they want to make sure it is helpful to them. For example, Seniors may not care about learning how to 
complete a hypothetical spreadsheet in Excel with all the various shortcuts; more likely they will want to know how 
to keep track of their particular farm records, download pictures of grandchildren or make labels in Word. A pre- 
session questionnaire can gather some of these expectations and interests ahead of time to enhance planning 
(Stevens, 2003); activities and concrete examples based on these interests will go a long way toward garnering buyin 
to the value of computing (Ross-Gordon, 2003). 

Each of these high-interest tasks should be taught using a single set of simple, step-by-step instructions, 
with a corresponding handout provided for later reference. For Seniors who wish to go into more depth with any of 
the tasks, they may do so in a different course or by speaking with the instructor after class or during an open lab 
time. By applying software procedures to real-life problems or interests, explaining why they should learn 
something, and encouraging them to ask questions. Seniors are more apt to gain confidence in being “self -directed” 
(Knowles et.al., 1998; Lawton, 2001). In addition, any resources that can assist them in working independently 
when away from the class are appreciated, such as the step-by-step job aides mentioned earlier as well as electronic 
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tutorials, information about where to get help (Seniors, 2004), and opportunities to work one-on-one with an 
instructor (Ross-Gordon, 2003). 

While older adults are often as knowledgeable and insightful as they have been earlier in their lives, their 
mental processing abilities are likely to have changed. Verbal memory peaks in the 60s and declines in the 80s, for 
example. Verbal ability plateaus between 40 and 60 and declines more steeply in the 70s and 80s. Math skills peak 
in the late 30s and severely decline in the 80s. As a result, Seniors who are in their mid -60s or older process 
information more slowly since they have less working memory. Their ability to shift judgments when given 
additional information is limited, as well, and it is more difficult for them to read between the lines. Seniors may 
also face increased difficulty in focusing on specific information and eliminating distractions; as a result, they are 
able to eliminate choices or possibilities more quickly than younger adults and often analyze information while 
making a decision (Stevens, 2003). These variables suggest a variety of instructional strategies that can promote 
learning and a sense of efficacy in a Senior audience; 

• Maintain a small instructor-learner ratio (Seniors, 2004). 

• Utilize a slower pace when speaking (Stevens, 2003; Morrow, 2002), explaining the purpose of a task or 
why they need to learn it (Redding, Eisenman, & Rugolo, 1998). 

• Employ a slower pace when doing computer work to allow for mouse or trackball coordination (Seniors, 
2004). 

• Present one task and then practice, before advancing to a new task. 

• Use stories to illustrate concrete examples (Ross-Gordon, 2003; Stevens, 2003). 

• Encourage questions at any time (Seniors, 2004). 

• Provide repetition and summarize frequently (Morrow, 2002). 

• Provide opportunities to figure out things for themselves (Knowles, 1990). 

• Incorporate collaborative, problem-solving activities (Dixon & Gould, 1996; Cahoon, 1996); even 
encourage attendees to bring along a friend, spouse or neighbor to discuss the information with later 
(Stevens, 2003). 

• Provide follow-up after the course, or give a review during subsequent sessions (Stevens, 2003). 

In conclusion, it is important to offer computing classes designed specifically for Seniors, preferably ones 
at various levels for varying computing skills. Pre -session surveys or interviews that collect information about 
computing skills, physical needs, interest, and expectations facilitate the planning for physical space 
accommodations and session activities. Instructional accommodations that are sensitive to the slower, but active 
mental processing of many Seniors will also go a long way toward promoting learning and enthusiasm for 
computing in general. As the market demand grows for America’s aging population, learning providers will find 
attention to these adaptations important in attracting Seniors to class offerings and in promoting retention and repeat 
registrations. 
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Abstract 

This paper considers student use of a web-based discussion forum in a second year, non-major Biology 
course. The authors discuss how meaningful participation in the forum is a form of public writing and may be an 
indicator of overall student success in the course. The authors also discuss how this success in the course is not tied 
to the students’ previous performance at the post-secondary level. 

Introduction 

Over the past decade, the researchers have been investigating the effects of student participation in 
electronic messaging. This investigation has evolved as both the technology has changed and the research trends in 
the field have adjusted to reflect both these new technologies and current theories behind learning in technology 
enhanced environments. As the focus of this investigation has evolved, it has shifted on the question of whether 
students’ participation in a class web-based discussion forum has an effect on their final course grade. 

This article reviews the evolution of this study and explores the research question based upon the nature of 
the participation and the individual abilities of the student. 

Literature Review 

Berge and Collins (1993) indicated that the main benefits of web-based discussion forums were the 
convenience for students, the time and place independence that it created for students, and the potential for students 
to become part of an online learning community. In addition to these primary benefits, there has also been 
considerable research into whether or not students participation in asynchronous means of communication, such as 
electronic mail and web-based discussion forums, have an affect on students’ performance. 

In 1989, Slovacek reported that "there appeared to [be] a positive correlation between students' use of e- 
rnail to augment normal in-class communication with their instructors and final course grades [specifically] that each 
e-mail message initiated by the students was associated with a 1.781 point increase in final course grade on average” 
(pp. 113-114). However, Collins (2000a) reported little difference between the final course scores of e-mail users 
and non-users. It was reported in that same study that a positive relationship existed between the level of Web forum 
use and final course scores . 

This distinction may be explained by Piirto (1998), who reported that approximately half of the students 
surveyed responded "never” or "not often," when asked if they proofread and/or edited their electronic mail. This 
was compared to 90% of students who responded that they proofread and /or edited their written documents "every 
time" or "most of the time.” (p. 28) According to Piirto, the level of care that university students place into their 
composing of an electronic mail message was very low. This was supported by Collins and Barbour (2001a), which 
speculated that while e-mail messages are often short messages about non-content queries which are “private” and 
only for the instructor's eyes, postings to the Web forum are “public” and open to the scrutiny of all class members. 
Students are more likely to be careful and deliberate about what they write on the web forum because they are for 
public consumption (p. 8). 

The careful and deliberate writing by students having an effect on student performance is supported by 
Ambron (1987), who reported “student response [was] extremely favourable; ... most mentioned the value of 
writing in helping them understand [the subject.]” (p. 266). Moore (1993) reported that “learning improves ... when 
writing assignments are complemented with instruction about how to use writing as a tool to learn [a subject.]” (p. 
217). In addition, Chickering and Gamson (1987) reported that one of the seven principles for good undergraduate 
education was that students “must talk about what they are learning [and] write about it” (p. 5). They also detailed 
that interaction between students and the professor and between students and other students is a key mechanism in 
enhancing learning. This is further supported by Moore (1994), who reports that people learn as they write, ideas 
form as pen hits paper and that writing about a subject is a way of knowing the subject because writing creates 
meaning. 
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This line of inquiry is supported by the research into microthemes. “Microthemes are a special kind of 
student writing whose length is strictly limited, usually to 150-400 words.” These written assignments have been 
used in many larger undergraduate classes in the hard sciences. “Microthemes are marked for their ideas rather than 
their grammar and spelling” (Collins, 2004, p. 7). Over a four year period, Collins (2000) found that there was an 
overall mean gain of 0.67% in student scores between those students who completed microthemes and those 
students who completed a term paper. In addition, the average test scores of students who completed 9-11 
microthemes was 13.2% than the average test scores of students who completed only 1-6 microthemes. 

The Study 

The courses considered in this study are Biology 2040 - Modern Biology and Human Society I (Human 
Biology) and Biology 2041 - Modern Biology and Human Society II (Environmental Science), large enrolment 
second-year, non-major Biology courses which were regularly offered in on-campus lecture, off-campus 
correspondence and Web-based formats. During the period 1994-2001 , student contributions to a discussion forum 
and e-mail messages sent to the instructor were collected. 

This study initially began as a professor “searching for a way to improve student-student and student- 
professor interaction in a large second year biology course . . . [that had] increased from 40, when [he] first taught it, 
to over 170 students in the 1995 winter semested' (Collins, 1995a, p. 1). The first system that was utilized for this 
project was a text -based environment, “DEC Notes would allow students to post and reply to notices, or to just read 
notes and their responses, although they could still contact [the professor]” (Collins, 1997, p. 189). Research into 
this system was focused upon how the students used the bulletin board, considering rnanyof the variables raised by 
Berge and Collins (1993). For example, Collins (1995b) stated that one of the perceived benefits of the system was 
“its availability around-the-clock so that students can ask questions, make comments, etc. at any time that suits them 
rather than having to wait for class or an opportunity to talk to the professor” (p. 189). Collins (1998) reported that 
“two thirds of the student use was for on-going discussions of a variety of topics such as the ethics of genetic 
engineering, the right to die, AIDS and youth, and the Red Cross and blood donations” (p. 80). 

This level of analysis was eventually expanded to include comparisons between different methods of 
delivery (i.e., correspondence, web-based and on-campus lecture) of the same course by the same professor. Collins 
(2000a) reported “that the correspondence section achieved the higher mean final scores in three of the four 
semesters while the Web course achieved the higher mean score in only one semester” (pp. 22-23). Additionally, 
Collins and Barbour (2001a) reported that “the results of an analysis of final course scores indicate that students in 
the Web-based sections generally perform at a lower level than those in the traditional classroom format, and these 
in turn perform at a lower level than students in the correspondence sections” (p. 310). At the time, these findings 
were in marked contrast to other similar studies (Navarro and Shoemaker 1999; Wideman and Owston 1999) which 
found that students in Web-based courses did better than those in both on -campus and correspondence courses. For 
example, Wideman and Owston (1999) reported that "on the whole students in Internet and in-class courses scored 
significantly higher than their counterparts in correspondence courses, although no significant difference was found 
between Internet and in-class students" (p. 2). 

Collins (2000b) marked the next phase of this study, when the authors began to consider the effect of 
participation in the Web forum upon a students’ performance in the course. “There [did] not appear to be a 
relationship between e-mail use and final course score” (p. 7). However, there appeared to be a positive relationship 
between Web forum use and final course score. Collins and Barbour (2001b) reported 

only ‘A’s were very frequent users, and only ‘A’s and 'B’s were frequent users. Only about one-third 
of ‘C’s, 'D’s and ‘F’s were infrequent users while two-thirds made no use of the Web forum. Students 
achieving an ‘A’ in the course were much more likely to be Web forum users (21 of 42) than 'B's (12 
of 29), who, in turn were more likely to be users than ‘C’s, ‘D’s, and ‘F’s (only 7 of 20). (p. 7) 

This relationship was further explored in Barbour and Collins (2002a, 2002b, & 2003a). 

However, this exploration led to the question of whether “higher levels of motivation or scholastic 
achievement may also lead some students to participate in electronic messaging more than others” (Althaus, 1996, p. 
14). It also presented another “question of whether it is the act of writing which accounts for this enhanced learning 
or whether that interaction be meaningfully based upon the content area” (Barbour & Collins, in press). 

Data and Findings 

As it was mentioned above, earlier studies have indicated a relationship between frequency of use of the 
web forum and final letter grade, as is summarized in Table 1. 
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Table 1 - Frequency of use of the web forum and final letter grade from 1997 academic year 





Course grade 


Level of use 


A 


B 


C 


D 


F 


Very frequent 


2 


0 


0 


0 


0 


Frequ\ent 


1 


2 


0 


0 


0 


Infrequent 


18 


10 


4 


1 


2 


None 


21 


17 


7 


2 


4 


Totals 


42 


29 


11 


3 


6 



However, this still does not address the question of whether it is interaction or the act of writing which 
accounts for this enhanced learning or should that interaction be meaningful based upon the content area. In order to 
answer the above question each student message was assigned a value based on the following scale. 

0 - No content basis 

1 - Administrative 

2- Content -based question or message 
3 - Content -based question or message with brief explanation 
4- Content-based question or message with substantial, but incomplete explanation 
5 - Content-based question or message with complete or near complete explanation. 

For this analysis, it was determined that the Spring 1999 semester of Biology 2040 was the only class that had 
enough messages to provide an adequate sample for this scale to be utilized effectively. 

While, the vast majority of messages that were posted to the web forum were rated in the lower three 
categories, with approximately 55% of the posts being assigned a rating of 0 or 1, there were five posts assigned a 
rating of 4 or 5. The majority of the messages analyzed were of an administrative nature, such as questions about the 
timing or format of assignments or exams, however, 46% of the messages did have some content-basis (i.e., were 
rated 2 or higher). When the scale value for each message is averaged on a student by student basis (e.g.. Student 1 1 
posted eight messages which were rated 2/4/2/2/1/1/3/1 would fall in the 1.51-2.00 range as the average value of 
these messages is 2), the following table is produced. 



Table 2 - Value of use of the web forum and final letter grade 





Course grade 


Value of use 


A 


B 


C 


D 


F 


2.01-2.5 


1 


0 


0 


0 


0 


1.51-2.0 


3 


2 


1 


0 


0 


1.01-1.5 


1 


0 


1 


1 


0 


0.51-1.0 


2 


1 


2 


0 


0 


0-0.5 


0 


0 


0 


0 


0 


Didn’t use web forum 


1 


3 


0 


0 


3 


Totals 


8 


6 


4 


1 


3 



This table indicates a similar pattern to the one found earlier by Barbour and Collins (2003b). The only 
student who had an average message value of higher than 2.0 scored an “A” in the course. The majority (5 out of 6) 
students who had average message values of 1.51 to 2.0 scored an “A” or a “B”. The only students who scored an 
“F” in the course did not use the web forum at all. 

The next data set addresses the issue raised by Althaus (1996), whether students do well because they 
participate more in the Web forum or whether stronger students are simply the ones who participant the most in the 
Web forum. The table below provides the difference between the student’s overall average at the university and the 
student’s average in the course based upon their level of participation in the Web forum. 



Table 3 - Students mean adjusted score based upon level of web forum usage 



Level of web forum usage 


Number of students 


Mean Adjusted Score 


None 


8 


-1.99 


Low 


7 


+0.10 


Medium 


4 


+8.25 
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High 



5 



+15.40 



As is indicated in the table, the eight students who did not use the Web forum at all had an average in 
Biology 2040 that was 1.99 percent less than their overall university average. However, the five students who were 
high users of the Web forum had an average in Biology 2040 that was 15.40 percent higher than their overall 
university average. 

Barbour and Collins (2003b) “indicated that there existed a positive, but not conclusive, relationship 
between the number of times students posted to the Web forum and the grade that the student received. This study 
has found similar results, not solely based upon simply interaction, but on meaningful, content-based interaction.” 
The data presented above illustrates that in addition to there being a positive relationship between students’ 
meaningful content-based participation in a Web forum and their final course grade, the relationship does not appear 
to be dependent upon the students’ higher levels of motivation or scholastic achievement. This analysis is also 
supported by the findings of Wu and Hiltz (2003). Wu and Hiltz found that “students felt that they learned a great 
deal from their peers through online discussion. . . [and that] online discussion increased their learning quality” (p. 
691). 



Conclusion 

Initial studies into the relationship between students ’ participation in electronic messaging and students’ 
final course grades indicated that there was a positive relationship between students participation in the Web forum 
and their final course grade, but no relationship between the use of e-mail and their final course grade. Later studies 
found that it wasn’t simply participation in the Web forum, but meaningful content-based participation that also 
showed a positive relationship to students’ final course grades. Based upon these findings, the researchers 
speculated the public act of writing in a web-based discussion forum had a positive affect on student performance. 

The data presented in this article indicates that in addition to these earlier findings, the researchers 
speculation appears to be correct, in that students do well because they participate more in the Web forum, as 
opposed to the notion that stronger students are simply the ones who participant the most in the Web forum. 
However, it should be noted that from five years, representing dozens of classes, there was only one class in which 
there was enough student participation in the Web forum to conduct this analysis. In addition, this one class only 
represented twenty-four different students (compared to the over one thousand students who have taken Biology 
2040 or 2041 during the period studied by the researchers. 
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Abstract 

As computer interfaces emerge as an instructional medium, instructors transitioning from the classroom 
continue to bear the burden of designing effective instruction. The medium of the computer interface, and the kinds 
of learning and interactive possibilities it affords, presumably changes the delivery of learner-centered instruction. 
Strategically, teachers not only need instructional design ability, but they also need competence with human- 
computer interaction design. In addition, instructors and instructional designers need to be able to bring these two 
domains together, if they are to create truly learner-centered instruction using new media. This article focuses on 
how a team comprising an instructor and multimedia instructional developers collaborated to create a distance 
learning environment for a graduate course in public health. The authors will describe the workflow that the team 
used, focusing specifically on the integration between instructional design and human-computer interaction design 
frameworks as well as how the team approached design issues by incorporating various HC1 theories. The authors 
will also show what insights the team had after a year and some of the strategic changes it made in light of them for 
the next. 
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Background 

In 2001, the Department of Applied Health Science in the School of Health Physical Education and 
Recreation [HPER] at Indiana University began utilizing information technology in all of its graduate level courses, 
as a part of its eventual goal to implement a Public Health certificate in a distributed education format. The initiative 
combines a mandate from the department, faculty financial support from School administration (in the form of a 
competitive stipend award), and instructional consultation and development support from the School’s pedagogy 
and instructional technology support office. The combination of human and financial resources into a shared vision 
has been critical to the initiative’s success. 

“Program Planning,” a 500 -level course, was the first to be implemented in a distance learning format, 
which the department believed would allow students to spend more time engaged in public health practice and 
service learning activities. Also, by delivering the course via distance learning, the course would be more attractive 
to public health professionals and other non-degree students throughout Indiana. In other words, the choice to use 
the distance learning format stemmed from two sources, one educational (emphasis on professional practice over 
classroom work) and the other merely pragmatic (ability to meet the needs of a geographically diverse group). 

The ADDIE Model for Graduate Public Health Education: 

An Iterative Approach 

The course development team used the ADDIE (Analyze, Design, Develop, Implement, and Evaluate) 
model of instructional design to manage the course development process. This process included student/user 
profiling, the formulation of teaching and learning objectives, constructing a blueprint for targeted instructional 
tasks, designing an attractive user interface, development of interaction designs, production of identified learning 
activities using various multimedia development tools and appropriate programming languages, learning application 
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testing and usability studies, and finally the evaluation of the quality and effectiveness of both the instructional 
design and the learning applications. 

Analyze: Learner Analysis and User Modeling 

Effective learning application design begins with learner/user research. For our team, this occurred on two 
levels: the instructional design level and the HCI level. For the former, we were concerned with what the students 
would already know, what they were there to learn, and so on. For the latter, we were concerned with their level of 
comfort with technology and their overall experience engaging with high-level work in a computer environment. 

Mike Kuniavsky, an HCI specialist, defines “user research” as a process that consists of “a consistent, 
rapid, controlled, and thorough method of examining the users’ perspective” (Kuniavsky, 2003). The practice of a 
successful user-centered design, which enables designers to take into account the users’ perspectives and 
ethnographical research during the design process, results in user experiences that are engaging, effective, efficient, 
and positive for users (Garrett, 2003). In addition to understanding basic information about the students (i.e., 
graduate students between the age of 25 and 40 in public health program) and what they need to learn (i.e., the 
process of developing public health interventions in order to positively affect the health-related behaviors of 
individuals, groups, and communities), the team devoted special attention to identifying the users’ computing 
inclinations as well as their perception of the use of distance learning as a learning format. 

The course design team implemented a Web-based survey on potential users at this stage to obtain a more 
accurate computing user profile for the learning application. 





Technology Inclination 


Hours per day spent using computers 




Hours per day spent using the Internet 




Personally own a computer 




Preference of location using computer 






Distance Learning 


Prior experience with distance learning 




Perception of using distance learning format for 
this course 




Motivation of finishing different distance 
learning course components 




Most comfortable format for course assignment 



Table 1: Sample Questions for User Profile Survey 



Through the study of behaviors and motivations of real people, the user profile survey provided us with a 
descriptive model of the users (i.e., their social and physical environments, their relationships with one another, etc.) 
and what they hope to accomplish in class. The survey indicated that the majority of the prospective learners had 
prior experience with distance learning format, over the Internet or via CD-ROM; more than 50% of the participants 
indicated the highest level of excitement about the prospect of taking the course in a distance learning format, and 
only 15% expressed concern. Most of the users owned a personal computer and felt comfortable using it for 
communication, learning, and other aspects of their lives. The ethnographic nature of the survey made it possible for 
the course development team to develop user models, or “composite archetypes” (Cooper, 2003), and create a 
context for potential learners (Beyer & Holtzblatt, 1998), which guided the course design team’s decision in 
translating learner profiles into an overarching design framework for the course, especially in areas such as course 
components, learning tasks, and interaction design. 

Design: A Two-Way Approach to Transcoding Learning Content to Bytes 

Scholars of media studies from McLuhan (1964) to Manovich (2001) have challenged the notion that we 
can separate the message (in this case, instructional content) from the medium (in this case, from the classroom or 
from a multimedia CD-ROM). Instead, they argue that the medium itself profoundly shapes and even constitutes the 
message. Others posit that the “embodiment” of information in a medium affects cognition as well: “embodiment 
makes clear that thought is a much broader cognitive function depending for its specificities on the embodied form 
enacting it” (Hayles, 1999, p.xiv). Manovich calls the mutual interactions that occur when content is expressed in 
digital media “transcoding,” which is the encounter between the “culture” and “computer” layers. If these critics are 
right, then it follows that simply translating instructional design intended for the classroom to instructional design 
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intended for Web- or CD-ROM -based learning is unlikely to succeed. The team’s goal, therefore, was to integrate 
HC1 and media theory into the systematic instructional design process (as described by Gagne, Briggs, & Wager 
[1992] and Smith and Regan [1993]) from the beginning. Instructional design entailed media design. 

The “Program Planning” course for public health was in part based on competencies outlined by the 
Council of Education for Public Health. One of the primary instructional goals of the course was to ensure that 
students developed these competencies. Two course components were identified to fulfill this goal: unit lectures that 
explain key concepts and principles of program planning, and interactive, skills -based application exercises. Where 
the lectures were to introduce concepts, the exercises would enable students to apply those concepts in scenarios 
based on real-world situations, providing students with the opportunity to practice problem-solving techniques 
(Gagne, Briggs, & Wager, 1992). The performance-based scenarios supported the practical orientation of the course, 
bringing about immediate and successful applications of the course content (Kindley, 2002). While it has been 
suggested that instructor-centered presentation of materials should be used sparingly (e.g., Crowl et ah, 1997), given 
the nature of the course and the delivery format, the instructor opted for initiating each learning unit with a topic - 
based, content -intensive lecture that guided students through the learning process. Nonetheless, these lectures were 
but a part of a much larger, and more diverse whole, which included interactive exercises, practice questions, and a 
learning community. 

The instructor was active in building a dynamic learning community that engaged a diverse range of 
students throughout the course in order to foster critical thinking skills through the weekly peer-discussion of current 
public health issues. The instructor was careful in reinforcing the targeted learning objectives of the week through 
online discussion forums that not only provided the appropriate context for the discussions, but which also 
encouraged in-depth dialogue on current topics of public health challenges. Scheduled events and regular office 
hours were conducted in the format of real-time chat to provide students with more opportunities for instructor- 
learner and learner- learner interactions as well as a sense of immediacy and presence (Kim, 2000). 

With the broad parameters of the instructional and interaction design established, the team next turned its 
attention to translating these general plans into a concrete learning application. At this stage, the findings of the user 
studies were particularly helpful, as the team engaged in several pre-production design preparations prior to coding: 
paper prototypes (Boling & Frick, 1997; Beyer & Holtzblatt, 1998; Snyder, 2003), information architecture 
(Wodtke, 2003), and storyboards, all with an aim to obtain cost-effective (both in terms of time and effort) user 
feedback to inform the directions of both interface and interaction designs. These activities enabled the course 
development team to render visually what the users would see and do and articulate what the users would 
experience. Prototyping carries many benefits: it engages different stakeholders (both the designers as well as the 
users) in the active participation throughout the design process; it enables an effective iterative design process until 
the design concept is stabilized so as to minimize the squandering of resources. All told, the empirical emphasis 
placed upon pre -production activities such as paper prototypes, storyboards, or information architecture provided 
more reliable guidelines for the course development team for interaction design (Dillon & Zhu, 1997). 

Assessing the effectiveness of the prototypes again required the team to gather evidence concerning the 
prototypes as learning applications and also the prototypes as software applications. The team looked for a 
combination of both usability goals (effectiveness, efficiency, safety, utility, learnability, and memorability) and 
user experience goals (satisfaction, enjoyment, fun, entertainment, helpfulness, motivation, aesthetics, reward, etc.) 
(Reece, et ah, 2002). In particular, the course development team followed four interaction design imperatives as 
suggested by Reimann, Dubberly, Goodwin, Fore, & Korkan (cited in Cooper & Reimann, 2003): 



Design Imperatives 


Definition 


Goals 


Ethical Design 


Considerate, helpful 


■ Do no harm 

■ Improve human situations 


Purposeful Design 


Useful, usable 


■ Help users achieve their goals and 
aspirations 

■ Accommodate user contexts and 
capacities 


Pragmatic Design 


Viable, feasible 


■ Help commissioning organizations 
achieve their goals 

■ Accommodate business and technical 
requirements 


Elegant Design 


Efficient, artful, 
affective 


■ Represent the simplest complete 
solution 
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■ Possess internal (self-revealing, 
understandable) coherence 

■ Appropriately accommodate and 

stimulate cognition and emotion 

Table 2: Interaction Design Imperatives 

As with many user interface and interaction designs, the learning application for the “Program Planning” 
course is based on the principle of direct manipulation of graphic objects (e.g., buttons, icons, function controls, 
etc.) on the screen (Cooper & Reimann 2003). The incorporation of direct manipulation in design enables the visual 
representation of manipulatable objects, replacing the user’s reading of text with the user’s own physical actions in 
that dataspace, and above all, the immediate demonstration of the result of the manipulation (Shneiderman cited in 
Cooper & Reimann, 2003). Direct manipulation helps move the student out of the comparatively passive role of 
reader and into the more performative role of doer. Abundant feedback was also important: given the intrinsic 
learner-directedness of the CD-ROM as a delivery format, the course development team felt a sense of urgency to 
design a learning application that frequently provides immediate and adequate visual feedback (so as to minimize 
the feelings of frustration, confusion, or being overwhelmed). Interactive graphical interfaces change the user 
experience from computer as tool or prosthesis to computer as space or environment (Johnson 1997). The team felt 
that the spatial metaphor was consistent with the instructional design emphasis on practice and performance. 

Development 

To ensure the timely delivery of the learning application and allow ample time for evaluation and redesign, 
it is important to construct a realistic project schedule that clearly identifies a list of milestones and concrete 
deliverables such as content, graphics, audio, video, programming snippets, and so on (Lee & Owens, 2000). To 
facilitate the managing and tracking of development progress and enhance the communication and collaboration 
among team members, in addition to regular project meetings, the course development team also used a Web-based 
project management tool, PHPCollab, to facilitate the development process. Because it was Web-based, the tool 
could be accessed via any browser connected to the Internet, which was convenient, because our team was often 
spread across different locations and operating systems. The project management tool enabled the team to monitor 
development activities and make necessary adjustments to development efforts in general and development tasks in 
particular. Image 1 and 2 illustrate the interface of this Web-based project management tool. 

Implementation 

One major challenge the course development team faced was to find a balanced, robust distance learning 
environment to meet the diverse learning objectives of the course. Knowing that many early efforts at distance 
learning were less than successful, in part because people tried to replicate the classroom experience too literally 
(Stevens-Long & Crowell [2002]), we tried, in the words of Siegel & Kirkley (1997), to be “computer imaginative”; 
that is, we strived to create learning designs that uniquely took advantage of the computing medium. Specifically, 
the team took advantage of its multimedia and asynchronous communication capabilities. 

After two versions of design and refinement, the finished integrated learning application included the 
following components: 

Multimedia Course CD. Blending video, audio, and user interaction, the course CD, authored in Macromedia 
Flash, served as the primary learning tool for this course. The Program Planning course was, compared to other 
courses, content-centered and skills -based. We felt that a multimedia CD provided diverse means of enabling 
students to engage and interact with, rather than simply read, course content. 

The course CD was divided into 10 learning units. Each unit consisted of the following four components: 

1) Lecture: An introductory video clip of the instructor providing an overview of the unit objectives, followed by 
unit content, in the form of audio -narrated slides with text, graphs, figures, and animations to facilitate learning. 2) 
Application exercises: Various interactive tasks that students perform to apply what they learned in the lecture. 
Completed exercises are captured and stored in the learning system for the duration of the session, making it 
possible for students to compare their work with solutions recommended by the instructor. 3) Self-test: An ungraded 
quiz that enables students to assess how well they accomplished the stated learning objectives. Upon completion, the 
correct answers and explanations are displayed side-by-side with the students’ answers. 4) Resources: The instructor 
provided additional resources, including articles, Web sites, and research findings relevant to the unit. Images 3-7 
show the course CD in its first generation, and images 8-13 show it in its second iteration. 
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The Internet Component. The course CD is strong at learner-to-content interaction, but working through a CD on a 
computer is an intrinsically solitary experience. Yet much of public health education is collaborative. To embed 
meaningful human collaboration into the course, we also integrated Oncourse, a Blackboard-like course 
management system as a shared learning space to foster collaborative learning. Students used its synchronous and 
asynchronous, formal and informal communication features to collaborate with their peers. 

In addition, Oncourse was used for web-based surveys and exams throughout the semester to assess student 

learning. 

Face-to-Face Interaction. Based on student feedback and a series of assessments conducted in 2002, in the second 
year of offering the course, the instructor built in bi-weekly and monthly face-to-face learning activities in 2003 to 
increase students’ access to instructor and fellow students. The instructor-guided activities consisted of site visits to 
community health centers, mock program planning and promotion development, debates on public health program- 
planning issues, public health participant/provider matching games and so on. As the course has evolved, the 
instructor’s role has gone from traditional office hours and email availability to guiding/modeling real-world health 
program planning problem-solving, blurring the distinction between instructor and students. 

Finally, to ensure long-term improvement of the project, the instructor and development team actively 
involve students to in the development and revision of the learning tool itself. This embedded recursiveness ensures 
that the team had opportunity to learn from the course and improve it in future iterations. The embedded evaluation 
process for the tool is two -fold: 

Ongoing Conversation about Technology and Health Promotion. Students are encouraged to engage in the ongoing 
discussion about their impressions of the potential benefits and limitations of information technology in the field of 
public health program planning and promotion. Special attention is devoted to the implications for different cultural 
groups and/or those with limited access to and/or experience with technology. The discourse not only raises the 
issue to the attention of all who participated, it also shapes the future offerings of the course. For example, some 
students noticed the lack of diversity in the images we used, and we were able to make appropriate changes based on 
their feedback. 

Learning and Technology Component Suggestions and Development. In an effort to involve users in the ongoing 
refinement and (re)designing effort of the course, the course development team adopted the “participatory design’’ 
approach (Beyer & Holtzblatt, 1998; Shneiderman, 1998) to bring about more accurate information about user 
inclination and needs, and above all, to increase user acceptance of the learning application. Students are required to 
critique all aspects of the learning application individually and provide suggestions on how best to improve the tool 
in both the pedagogy and technology areas. As part of the course requirement, students need to design at least one 
application exercise for a particular learning unit and develop 5 questions to contribute to the self-test question bank. 
In addition, the students are paired in groups to envision a relevant multimedia piece or interactive learning activities 
that target a particular aspect of public health program planning. Throughout the process, the course development 
team works with students closely to provide consultations, training, and necessary resources. At the end of the 
semester, not only do the students improve technically and know what it takes to integrate technology in the field, 
these efforts also result in a library of student-centered learning applications that the instructor and the development 
team can use in the ongoing revision of the learning tool, with a diversity that supports a broader range of learning 
styles. 

Evaluation 

Evaluation of instruction is important in any instructional setting, but it was especially important for this 
project, because this was the first offering of this class in a distance learning format; it was the first time the 
instructor had designed for a distance learning format; and we brought together an interdisciplinary collection of 
theories (instructional design, HCI, and media studies) and experts (the instructor, instructional designers, 
multimedia authors, and programmers). To ensure adequate evaluation, the course development team introduced 
evaluation at every stage of the process, from early prototypes through end-of-course evaluations. These evaluations 
reflected our interdisciplinary approach, seeking to learn not only about how the participants learned, but also how 
their attitudes toward technology changed. 

Using various evaluation methodologies such as “Quick and Dirty” evaluation and usability testing (Reece 
et ah, 2002), the course development team focused on several key issues in particular during the evaluation stage: 
learners’ ability to comprehend and apply knowledge, comparison of learners’ attitudes towards distance learning 
before and after the course, the time needed for the learners to prepare for and complete each learning unit (vs. that 
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for the same class offered in the traditional setting), learners’ perception of instructor availability, learners (users) 
and learning tasks; learners’/users’ interaction with the learning application, etc. 

For example, the course development team learned from the first year of the offering that students felt 
disappointed about the lack of interaction with each other and with the instructor, and we used a blended learning 
approach and introduced regular meetings (once or twice per month) with the instructor as well as many mini group 
projects. While most students felt comfortable using course CD and the Internet, at the beginning of each offering, 
we designed a technology orientation session to introduce all the components to ensure technology does not become 
a hindrance in students’ learning process. 



Adaptation 

In spring 2004, the instructor and the development team began to repurpose this learning application for all 
public health providers in Indiana. The content that is suitable mainly for higher education, as opposed to the more 
generalized needs of all public health providers in the state, was removed and the interface was modified for the 
different user group. The final result represents how an integrated learning tool designed for higher education can 
also be used to meet the needs of practitioners. In spring 2004, hundreds of this modified version of the learning 
application were distributed across the state. Images 14 to 18 show the pubic health providers’ version of the 
learning application. 



Summary 

The “Program Planning’’ project demonstrates it is essential to integrate sound pedagogy practices (i.e., 
appropriate instructional design principles, learning strategies, etc.) as well as HCI practices (i.e., user modeling, 
usability, prototyping, evaluation, etc.) and media theory to ensure a successful learning experience in the domain of 
distance learning. The multidisciplinary, team-based methodology in approaching both instructional and system 
design challenges results in a more holistic final product that reflects the theories and practices of a number of 
relevant domains. 
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Image 1: Web-based Project Management Tool 
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Image 2: Web-based Project Management Tool 
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Image 3: Course CD — The First Year 
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Image 4: Course CD — The First Year 
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Image 5: Course CD — The First Year 



37 











Image 6: Course CD — The First Year 
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Image 7: Course CD — The First Year 
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Image 8: Course CD — The Second Year 
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Image 9: Course CD — The Second Year 
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Image 10: Course CD — The Second Year 
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Image 11: Course CD — The Second Year 
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Image 12: Course CD — The Second Year 
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0 0 Google * 
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|v gt Search Web ^ 


Address Q E:\activeinformation\unit7.html 
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C501 Resources 



UNIT 7. Planning for Intervention Development 



A Guide for Making Print Documents Accessible to Persons with Disabilities ■a Accessible Print Docs.pdf 

Best Practices Home Page This searchable database contains over 1,000 proven solutions from more than 120 
countries to the common social, economic and environmental problems of an urbanizing world It demonstrates the 
practical ways in which communities, governments, and the private sector are working together to improve governance, 
eradicate poverty, provide access to shelter, land and basic services, protect the environment and support economic 
development.Best Practices Home Page is located at dil http://www.bestpractices.org/ 

Best Practices in Collaboration to Improve Health The California Wellness Foundation and the Health Care Forum 
partnered to develop the Emerging Best Practices in Collaboration to Improve Health: Creating Community Jazz 
workbook, designed to support community-based partnerships worldwide in the collaborative effort. This includes 
information about organizing the effort, convening the community, creating a shared visions, assessing current 
realities/trends, action planning, etc. Also included are four case studies and sample survey questions, capacity inventory, 
logic model, and action plans. Access Best Practices in Collaboration to Improve Health at r. ) http://www.act- 
toolkit. com/toolkit/toc.htm 

Beyond the Brochur e Beyond the Brochure.pdf 

Beyond the Stoop: Community Programs to Promote Youth The Institute of Medicine, of the National Academies, 
developed this document to address public policy for youth community programs. More information about this document 
can be found at the Institute of Medicine's web site or by looking at the publications listed at the National Academy Press 
site. 

o 41 http://www.iom. edu/ 

o -gH http://www.nap.edu/ 
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automatically. Once you begin each lecture, you can navigate through the lecture using 
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Teaching Online Teaching: Selecting Research-Based Instructional Strategies 

Brian J. Beatty 

San Francisco State University 

Abstract 

This paper describes the Social Interaction Online (SIO) tool, developed to help instructional designers 
and online educators develop effective online learning environments. Designers are encouraged to choose 
instructional strategies and activities (methods) while considering the specific conditions in their learning 
environment that are likely to impact the effectiveness of the methods they choose. Findings from existing case study 
data (30 cases from a previous study - Beatty, 2002) provides the set of initial instructional design guidance. The 
SIO tool was developed using standard web development and database software to create interactive data-driven 
web pages that show users how to choose effective instructional strategies for their online courses. Multiple 
applications of the SIO tool and situationalities framework approach are explained. The SIO tool supports and 
encourages users to contribute additional information ( experience-based data or research findings) which will 
expand and enhance the instructional design guidance as the website is used and matures. The SIO tool is available 
at http://online.sfsu.edu/~bjbeatty/sio/ 



Introduction 

This paper presents an interactive online tool. Social Interaction Online (SIO), to support educators and 
instructional designers in selecting specific instructional strategies to use in their online courses. Online learning has 
emerged as one of the most important new areas for research and development in the field of instructional 
technology (NCES, 2001; WBEC, 2000). Using learning theories developed in both distance education and 
traditional classroom instructional settings, online learning educators (including instructional designers) are using 
new, technology-enhanced instructional methods and more traditional forms of instructional methods to incorporate 
social interaction in their online learning environments (Ahern & El-Hindi, 2000; Bonk & Kink, 1998; Gilbert & 
Moore, 1998; Harasim, 1990; Muirhead, 2000). 

Most existing design guidance for online learning environments does not address the conditions (an 
important part of the situationalities) that affect the selection of instructional methods, especially for methods that 
engage learning participants in social interaction. Prescriptive design guidance should include a discussion of the 
specific situationalities (learning goals, values, conditions, and outcomes) that lead to the choice of one or more 
instructional methods in preference to other methods (Reigeluth, 1999). In the research underlying this report 
(Beatty, 2002), I developed prescriptive design guidelines using a situationalities framework which demonstrates the 
usefulness of the framework for the further development of online instructional theory. Launched in summer 2004, 
the interactive website makes this framework available and useful to educators and instructional designers using 
online instructional methods. 



Social Interaction in Online Instruction 

Many educators and learning theorists consider learning to be a largely social process (Bruner, 1990; 
Dewey, 1897; Hutchins, 1996; Lave & Wenger, 1991; Rogoff, 1990; Salomon, 1993; Vygotsky, 1978; Wertsch, 
1997). While not all learning environments require an explicit social interaction element for effectiveness (e.g., self- 
paced tutorials and review guides), most online learning environments are designed to use some measure of social 
interaction in the learning approaches they implement. 

Defining social interaction 

Interaction has been described with many terms and classifications, often in very dissimilar ways. Different 
authors focus on dissimilar aspects of interaction or sometimes just use dissimilar terminology. Rose (1999) even 
goes as far as asserting that the concept of interactivity in the instructional technology literature is “a fragmented, 
inconsistent, and rather messy notion ...” (p. 48). The variation in the literature seems to bear witness to the 
"‘messiness” of the concept of interaction. 

Wagner (1994) takes a systemic approach in her development of a functional definition of interaction. She 
includes the contexts of instructional delivery, instructional design, instructional theory, and learning theory in her 
attempt to establish conceptual parameters for the function of interaction. Hiltz (1995) reports that if an instructor 
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can facilitate meaningful, engaging cooperative group experiences online, students are likely to experience a greater 
sense of interaction than in a traditional face-to-face course. Abrami and Bures (1996) describe asynchronous, non- 
face-to-face interactions as “asocial” in general, but also consider collaborative interactions among students as 
essential factors in successful distance education. Feenberg and Bellman (1990) describe the importance of “social 
factors” in designing distance learning environments that use CMC technology. They consider the design of the 
social environment in distance learning as comparable in importance to the interior design of a face-to-face learning 
environment, meaning that the effort an educator takes in designing a classroom environment for social interaction 
(e.g., chairs arranged in small groups for collaboration or in a large circle for class discussion) should also be taken 
by an online educator. This may mean creating a unique virtual space for each collaborative group in a class, or 
creating a common discussion space for a whole -class discussion. 

Gunawardena and Zittle (1997) show that the degree to which a person is perceived as “real” in CMC, a 
concept also referred to as social presence, is a strong predictor of satisfaction in distance education. Gilbert and 
Moore (1998) describe an “interactivity taxonomy” for web-based learning environments, developed along the two 
factors: social interactivity and instructional interactivity. Walther (1996) posits that CMC technologies support 
impersonal, interpersonal, and “hyperpersonal” communication interactions. Walther describes hyperpersonal 
interactions as interactions with heightened levels (feelings) of intimacy, solidarity, and liking, which cannot be 
achieved through face-to-face interactions but can be experienced through CMC facilitated interactions. Yacci 
(2000) defines interactivity with four major attributes: the existence of a message loop, the completion of the 
message loop from the learner’s perspective - from and back to the learner, the provision of both content learning 
and affective benefits, and the need for mutually coherent messages in each interaction. Yacci points out the need 
for a common definition of interactivity, and provides the structural process definition as a starting point for future 
research. 

Previous literature reviews have focused: (a) on the building of effective interaction in distance education 
(Flottemesch, 2000), where online education is included only as a minor component in the reviewed research, and 
(b) on the use of student interaction of both a social and informational nature in online learning (Liaw & Huang, 
2000). Clearly, when writing about social interaction and designing socially interactive learning environments, a 
designer or educator must clarify his or her perspective on what “social interaction” means. For the purposes of this 
report, I define social interaction simply as “intentional communication between two or more participants in the 
learning environment.” 



Selecting Instructional Methods 

Beatty (2002) described the importance of understanding the relationships between types of situationalities 
(values, goals, methods, and conditions), such as those between values about learning and learning goals. Beatty 
reported relationships between associated pairs of situationality elements, since there is a sequential nature to design 
guidelines derived using this approach. For example, a designer’s values about learning directly influence the 
designer’s learning goals, which in turn directly influence the instructional methods the designer chooses fora 
particular learning setting. Similarly, the instructional conditions present in the learning environment directly affect 
the effectiveness of the selected instructional methods. 

Values and Goals 

A designer or educator’s strongly held, fundamental values about learning should influence the mo st 
important characteristics of a learning environment s/he designs. The educator’s learning goals are usually derived 
directly from their most important values about learning. For example, an educator who values interactive dialogue 
may state a learning goal of “Students learn how to learn through dialogue with each other.” In most cases, an 
educator’s learning goals reveal what the educator truly values about learning. For that reason, it is important to 
consider the relationship between learning goals and fundamental values about learning. 

Goals and Methods 

Hopefully, a designer or educator’s learning goals are derived directly from the educator’s most important 
values about learning, as explained above. Once the learning goals have been developed, the next logical step in 
designing instruction is to choose appropriate instructional methods. The instructional methods are the “vehicles” by 
which the designer’s learning goals for participants are met in the learning environment. For example, a designer 
who has established a learning goal of “Students will retain control over important aspects of their own learning.” 
should choose instructional methods that are likely to result in student control in the learning environment, such as 
“Students are required to choose a discussion topic and then prepare and moderate a weekly discussion on that topic 
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for the entire class.” It is therefore important to consider the relationship between learning goals and instructional 
methods. 



Methods and Conditions 

A designer’s learning goals directly influence the instructional methods s/he chooses for a learning 
environment, as explained above. In order to create the most effective learning environment, a designer should also 
consider the instructional conditions that affect the effectiveness of the chosen methods. These conditions must be 
met in order for the instructional methods to be effective. For example, a designer who is considering an 
instructional method from a category such as ‘‘Asynchronous Group Discussion”, should consider the instructional 
conditions that affect the effectiveness of this type of method, such as ‘‘Students must have the time available to 
participate frequently in class discussions.” If this condition is not met, for example when there is a heavy reading 
and writing load in a course, students may believe (or perceive) a participation requirement is just busy-work. If they 
had already written a paper on the discussion topic during the same week, they saw no need to discuss the topic 
online with their peers. (Case report Cl 10 from Beatty (2002); derived from Vrasidas, C., & Mclsaac, S. M. (1999)) 
Instructional methods chosen to meet particular learning goals are only effective when associated instructional 
conditions are met, so it is important to understand relationships between instructional methods and instructional 
conditions. Therefore, the instructional conditions present in the learning environment (including the participants) 
should be taken into account as instructional methods are chosen. 

Situationalities Framework 

Beatty 2002 presented a ‘‘situationalities framework” to show the conceptual relationships between 
situationalities such as instructional values, learning goals, instructional methods, and associated conditions. 

(See figure 1.) The Social Interaction Online (SIO) web-based tool helps designers and educators follow this 
framework as they select instructional methods for online environments. 
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Figure 1 . Situationalities Framework 



Phase I: Values and Goals Phase II: Goals and Methods Phase III: Methods and Conditions 
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To do: Consider instructional conditions that 
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influence the effectiveness of chosen 


goals based on these values. 




Multiple methods may be preferable. 
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expectations for participation. 



- METHOD 2 
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other 

~ METHOD 3 ^ ^ conditions 
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METHOD N 



other methods 
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Example: Student Motivation - Each 
group member must be accountable 
for their part of the group project. 



- CONDITION 2 
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Students must be willing to work 
with the same group of students 
for an entire semester. 



- CONDITION 3 
Example: Student Motivation - 
Peers must be willing to critique 
each other's work. 



CONDITION N 



other conditions 



54 








Using the Situationalities Framework 



General Instructions 

When using the situationalities framework approach to choosing instructional methods, the following six 
steps summarize the major tasks for the educator/instructional designer. 

1. Determine your learning values, related to social interaction in online learning, that you would like 
your learning environment to support. For example, do you value forming online learning 
community? Or, maybe you value students learning through interactive discourse. list these 
fundamental values in preparation for the next step. 

2. Identify the learning goals that you want students to achieve. These learning goals should reflect 
your previously determined values. It is important to list at least one learning goal for each value. If 
you are not very familiar with online instruction, you might need to consult resources . If, in step 1, 
you stated your values in terms of means - what students should be doing - you do not need to 
identify specific learning goals in this step. (In this case, you are ready for step 3.) 

3. List alternative instructional methods that will meet the learning goals you just identified. If you 
are not very familiar with online instruction, you might need to consult other resources . It’s 
important to consider several instructional methods from which to choose, since it is likely that not 
every method will fit the specific conditions (see Step 4.) that exist in your online learning 
environment. 

4. For each of the instructional methods you listed in step 3, assess the instructional conditions that 
should be met in order for the method to be effectively employed. List the instructional conditions 
next to each of the alternative instructional methods. This is perhaps the most important step in using 
the situationalities framework. If you try to simply implement instructional methods without 
assessing whether or not the necessary instructional conditions are met, your online instruction may 
not be very effective. 

5. Select specific instructional methods from the list of alternatives based on your assessment of 
instructional conditions. For maximum effectiveness, choose methods that have all their associated 
instructional conditions met. If an important instructional condition is not met for a particular 
method, you should either choose an alternative method, modify the method to take into account the 
deficient condition, or add a secondary instructional method that will create the necessary condition 
for the primary method. For example, if you would like students to engage in interactive 
asynchronous dialogue, but you assess that your students are not likely to synchronize their 
participation in online discussions appropriately on their own, your instructional method should 
attempt to structure this interaction for them. You might assign an initial post on a particular day of a 
week, with a second deadline, later in the week, for a reply to another student’s post, and a third 
deadline, also later in the week, for a reply to an original comment. 

6. Implement the instruction. During the course of instruction, you should frequently assess the 
effectiveness of the methods you’ve selected. If a method seems to be ineffective, consider whether 
important instructional conditions have not been met. It is very possible that there are unique, 
significant conditions associated with the method you are implementing in your specific setting - 
conditions which are not addressed by the guidance contained in the tables and appendices of this 
study. 

Using the Online Tool 

The online tool, Social Interaction Online (SIO), supports the designer following the general approach 
explained above in multiple ways. First, the designer may choose to view a list a instructional values which have 
been published in case reports over the past few years. After selecting either a category of value (such as, 
Collaboration) or a specific instructional value (such as. Online Collaborative Learning), the designer then views a 
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list of associated instructional (or learning) goals. After selecting either a category of goal (such as, Collaboration) 
or a specific instructional goal (such as. Students learn how to collaborate on learning tasks with their peers.), the 
designer then views a list of associated instructional methods. Each instructional method is linked to an explanation 
of conditions that affect the effectiveness of that particular method. The designer would consider important 
conditions as they select instructional methods. After selecting one or more instructional methods from the list 
provided, the tool creates a summary page that the designer can save, print, or email for further use. 

See Figure 2. 

A second way that the SIO tool supports designers is helpful when a designer would prefer to explore 
associated values, goals, methods and conditions holistically as they have been reported in cases. When this is the 
preferred approach, the designer can choose to view a list of cases included in the SIO dataset. Each case provides a 
full citation reference and a hyperlinked source or contact person, if possible. Clicking a case number opens a page 
which lists the case information again, the instructional values, instructional goals, and associated methods for each 
goal reported. For each instructional method, associated conditions and a brief explanation of how each listed 
condition affects the effectiveness of the method is provided. Using the SIO in this way allows a designer to focus in 
on methods and conditions specific to a particular case, which may be especially useful if the designer’s 
instructional context is similar to that of the original case author. 

See Figures 3 and 4. 

Besides providing guidance about selecting specific instructional methods for social interaction in online 
learning environments, the SIO website supports designers with rationale for considering instructional values and 
goals. Some designers neglect, often due to time and resource constraints, to reflect upon the type of learning 
environment they are creating with instructional methods and activities. The SIO supports reflective teaching and 
instructional design practice so that learners are engaged with purposive instruction designed to help them meet 
strategic learning goals aligned directly to meaningful instructional values. (See the SIO tool at 
http://online.sfsu.edu/~bibeattv/sio/ ) 



Conclusions 

The SIO can be an important tool for instructional designers and educators who design online learning 
environments that use social interaction. The comprehensive situationalities framework presented here (in Figure 1) 
and which forms the foundation of the SIO design guidance should help educators design online learning 
environments that use effective instructional methods aligned to their fundamental values about learning, learning 
goals, and which consider specific instructional conditions in a instructional setting. If an educator follows the 
design guidance and uses the SIO data resources, then the selected instructional methods should fit the specific 
conditions of the learning environment, and the selected methods should be effective. At the very least, learning 
should not be impaired by (unknowingly) unmet instructional conditions. If additional instructional conditions 
emerge during the instruction, resulting in ineffective instruction, the situationalities framework approach can help 
an educator modify his or her instructional approach to achieve instructional goals. 

One of the important results of the research underlying the SIO tool is the opportunity to apply findings 
from aggregated case reports. One of the significant limitations of the descriptive case study format is the general 
inability of a case report reader to transfer many of the specific findings from one case to their own situation due to 
the inherent contextualization of the case report’s findings. It is sometimes simply left up to the reader of the case 
study to identify relevant, useful findings that can be applied to their own specific case. While this works well for 
some cases, an author may be able to enhance the usefulness of her or his case report to other educators by including 
explicit discussion about instructional values, goals, methods and conditions. Further expansion and enhancement of 
the SIO tool (and the situationalities framework approach) relies upon descriptive case studies that include a 
discussion of each of the situationality elements (values, goals, and conditions), as well as instructional methods. 

One of the most significant limitations of the SIO tool is the constrained focus of case reports on strictly 
online learning settings. In order to limit the scope of the study to a manageable size, I did not consider cases of 
online learning environments that were designed to support, extend, or enhance face-to-face learning environments. 
These “hybrid” contexts are becoming more and more popular in online learning, and should be studied separately, 
in order to identify the effective instructional methods and important instructional conditions for that environment. 
Hybrid learning environments use many of the same technologies and instructional methods as “pure” online 
learning environments. However, since the participants in the learning generally meet regularly face-to-face in a 
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classroom, there may be important differences in the design and implementation of social interaction in the online 
component of the learning environment. 

Even though the SIO tool was developed with data derived from case reports in the academic environment, 
the tool should be of some use in corporate and military training environments which use social interaction in online 
learning. For example, conditions of learner motivation are relevant in every instruction situation, as are conditions 
of access, technical support, and skills and ability. What varies among different contexts, however, is the extent to 
which conditions related to learners, instructors, etc. are met in the specific context, and the extent to which sets of 
instructional methods should be varied, based on the different sets of existing conditions. Therefore, a designer or 
educator designing training in a corporate or military setting should find the study findings, including the 
situationalities framework, relevant to their work. 
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Additional Figures 



Figure 2. List of Conditions for Selected Method 



Selected Method: Provide several modes of CMC technology (discussion, chat, etc.) for small groups of 4-5 
students to use as they complete a collaborative project. 



3 Reported Conditions: 



Student Background 


Effectiveness 
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Experience using CMC technologies and familiarity 
with course content were helpful for collaboration. 
More experienced students implemented a more 
effective collaborative process. 


Students are experienced in 
distance teamwork and have had 
some experience in the content 
domain. 


Student Motivation 


Effectiveness 


Condition 




Teamwork was not effective when some team 
members lacked self-discipline and overall 
commitment to the collaborative project. 


Students must have enough self- 
discipline to manage their time 
effectively and be committed to 
completing the team project. 


Skills and 
Student Ability - 
Technical 


Effectiveness 


Condition 




When wide levels of technical skill existed among 
students in a group, collaboration was difficult, all 
available collaboration technologies were not be 
used, and the collaborative process was slow' and 
ineffective. 


Students in a group should have 
similar technical skill levels and 
be trained in the use of the 
various technologies. 
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Figure 3. Case List 



Case ID 


Citation 


C101 


Cifuentes, L., Murphy, K. L., Segur, R., & Kodali, S. (1997). Design considerations for computer 
conferences. Journal of Research on Computing in Education, 30(2), 177-201. 




Hvperlink: http: coe.tamu.edu -lcifuent vitae. php 


C102 


Poole, D. M. (2000). Student participation in a discussion-oriented online course: A case study. Journal 
of Research on Computing in Education, 33(2), 162-177. 




Hvperlink: http: www.csustan.edu advstd dawn da\vn.html 


C103 


McAlpine. I. (2000) Collaborative learning online. Distance Education, 21(1), 66-80. 




Hyperlink: 


C104 


Zhang, P. (1998). A case study on technology use in distance learning. Journal of Research on 
Computing in Education, 30(4), 398 □ 420. 


Hyperlink: 


C105 


Wegerif, R. 1998. The social dimension of asynchronous learning networks. Journal of Asynchronous 
Learning Networks 2 (1), pg 34-48. 


Hvperlink: http: www. aln.org publications ialn v2nl v2nl weeerif. asp 
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Figure 4. Case Report Detail 



013 Chester, A., & G Wynne, G. (1998). Online teaching: Encouraging collaboration through anonymity. Journal of 

Computer-Mediated Communication, 4(2). [Online ] Available: http: vww.ascusc.org jcmc voH issueJ chester.html 
Accessed Sep. 5, 2001. 

Value: Online Collaborative Learning ( Collaboration ) 

Goal: Students will learn how to become part of a learning community, (community) 

Method: Provide an online forum dedicated to non-content-focused 'social" discussion. 
(Asynchronous, Student-student, Social) 

Condition: Effectiveness: 

Students must value community among Students who personally valued "community" 
themselves and trust each other. used this space to get to know one another. 

(Student, Values) Students who did not value community did not 

use this discussion space. 

Method: Students interact online using aliases only - no real identities are used. 

(Asynchronous, Student-student, Discussion) 



Condition: 

Students must be willing to use an alias 
without revealing their true identity'. This 
may be especially challenging if they are 
co-located and face-to-face meetings are 
possible. (Student, Motivation) 



Effectiveness: 

Many students w'ho w'ere normally silent in class 
participated actively in online discussions. 
Students enjoyed getting to know' each other 
through aliases without the "normal" external 
complicating factors, such as gender, age, race, 
and appearance. 
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Introduction 

As the internet-based learning (IBL) market becomes increasingly global, understanding differing 
educational values and cultural expectations could provide an important competitive edge for providers (universities, 
publishing houses, and corporate training entities). How each of us determines “good” or “quality” instruction is to 
a large degree founded on what educational values we hold. These values are primarily shaped by (1) cultural 
norms, (2) the philosophy(s) of learning to which we adhere, and (3) our personal preferences for learning. When 
our educational values match those embedded in the course 2 the match-up contributes to our perception of it being a 
quality educational experience; conversely, when our educational values do not match those of a course, then 
dissatisfaction is likely to occur. 

It may take a student some time to discern the degree of match between personal values forjudging quality 
instruction and the instructional values in the course. Sometimes, savvy students familiar with their local 
educational delivery system can infer the teacher’s or instructional designer’s educational values quickly from how 
the syllabus is designed. Then based on that rough assessment students choose to stay enrolled or not, depending on 
how successful they think they might be in the course, thus avoiding situations with large value differences. 
However, not all students know the local culture well enough to be that insightful. When an international student 
attempts the value matching process across cultures, differences between their (home) local system and the 
(international) local system are magnified. The result can be students choosing courses in which they are more 
likely to perform poorly educational . 

As increasing numbers of international students choose to take IBL courses which are designed by 
instructors outside their country of origin, they need more help than local students in selecting courses in which they 
are likely to have a successful learning experience, hey will prefer one more congruent with their cultural 
expectations. (Tom Nickles, personal communication, May 12, 2004). Out-of-country students expect instruction 
delivered abroad to be different from what they would receive studying in their own country. They are more likely 
to be actively seeking new kinds of learning experiences taught in the local metaphor. According to Zamel & Spack 
(2004) students can adjust faster to taking courses in a different cultural presentation style if they realize they are 
joining a separate academic discourse community and they begin to try on the discourse of the new setting. 

Learning how to read the big picture of a course and seeing what’s shared and valued within the community helps 
them adjust better. 

Not all learners and instructors are aware of this difference in expectations. Therefore wherever significant 
differences might be expected, such as with diverse new learners to a department/cohort, a new student in graduate 
school, or moving to a new school in another state, similar value mismatches could be expected to exist. Learners 
whose value differences are likely to differ most from the local , should have the information and options that will 
allow them to choose courses that match their own educational values. 

The designer has the responsibility to make the courses educational values explicit in the course materials 
and it is the learner’s responsibility to understand themselves as learners and find out about the context from which 
the course originates. This paper recommends a new intercultural standard for expressing the instructional of a 
course through which designers (producers) and students (consumers) can clearly communicate the educational 
values to each other. It should be similar to that of food labeling. We believe that designers should make the values 
imbedded in the course visible to the learner in an advance syllabus or course description. Eight educational value 
differentials or factors can help us make a distinctive difference in how the learner perceives quality in instruction, 
will discuss how designer integrate the eight differentials in preparing instructional materials and apply strategies to 
match users to suitable courses. We conclude with two handy checklists of recommendations distilled from the 
research; one for low-context (North American or Western) instructional designers and one for high-context 
students. 
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Eight educational value differentials for IBL 

Through 30 years of collective cross-cultural educational experiences and a review of literature, the authors 
have determined that there are at least eight educational value differentials or factors which make a distinctive 
difference in how the learner perceives quality in instruction. The eight educational value differentials discussed in 
this paper are language, culture, technical infrastructure, local/global perspective, learning styles, reasoning patterns, 
high/low context, and social context. It is not possible to value everything equally. The competing demands on a 
limited set of resources influences where resources are allocated. These eight value differentials appear to be the 
primary pivot points around which major cultural differences in perception of quality instruction currently hinge. 

1. Language differential 

Differences in cultural values, mores, and practices, are heavily influenced by constructs of their native 
language. Every culture has a predominant language that may seem simple at first glance to its users, but each 
language empowers its speakers with the ability to converse, participate in life with a social identity, express a 
complex range of ideas verbally and non-verbally, and process time (Mayer, et ah, 2003). Language and culture are 
intertwined, and it is difficult to understand one well with out understanding the other, as new students of any given 
language soon discover. Just learning the words of a language is not enough. “Rather, language can serve as a bridge 
to facilitate a deeper understanding of culture” (Helmer & Eddy, 2003, p.35). Indeed, the social and economic 
divides are growing between speakers of certain languages as the process of globalization connects and yet separates 
certain nations or sub-cultures within nations (Friedman (2000). 

Such a divide is obvious between the academic, social, and economic growth of English speakers and the 
struggles of non-English speakers, for example. A recent study by a leading British linguist, David Graddol states, 
“In many parts of the world, English is now regarded as a basic skill, like computer skills, which children learn at an 
early age so they can study through English later”(Ward, 2004, p.6). He predicts that in the future, most people will 
speak more than one language and switch between languages forroutine tasks with the language that best suits their 
needs in any given situation. “English has become more than an optional lingua franca: it is now the required 
language of world empire: political, military, economic, and cultural” (Edge, 2004, p.35). 

Many international English-as-a-second -language (ESL) learners who take online courses find that their 
cultural orientations and second language abilities may magnify their problems at first as they attempt to complete 
IBL courses (Warschauer, 1999), but these can be overcome with increased use of online courses. Some university 
leaders and course designers may think that as long as their online course is in English, it is equally available to any 
student who speaks English. However, instructional designers preparing for a global audience would do well to 
remember in their needs analysis to choose an appropriate level of English for their international courses. Because 
there are currently more ESL learners in the world today than there are native English speakers (Mauranen, 2003; 
Ward, 2004), instructional designers and teachers need to express content simply and precisely in IBL courses. 

When designers know they will have both native and non -native speakers responding to the instructional 
discourse style, as much as possible they should create materials that are culturally neutral. This requires use of a 
simpler sentence structure and avoiding slang, colloquialisms, local humor, and local insider examples whenever 
possible. In the 21 st Century in general, IBL designers and instructors would do well to consider that in some ways 
they are always designing for a global audience. Warschauer (1999) reminds us that the Internet allows 
communication in hundreds or thousands of languages at the same time as evidenced by Internet discussion boards 
available in so many languages. He expects that people will use English on the Internet “for certain instrumental 
reasons”, as a tool, while they use their other languages in their daily lives (Warschauer, 1999, p.19). 

2. Cultural differential 

There are many ways to describe culture and cultural differences, and no one universal definition of culture 
exists because it exists everywhere, among all people in different ways. Peter Chinn observed, “Culture is so much 
an integral part of our life that it is often difficult to realize that there are different, but equally valid, ways of 
thinking, perceiving, and behaving” (Helmer and Eddy, 2003). Neuliep (2003) defines the essence of culture as “an 
accumulated pattern of values, beliefs, and behaviors shared by an identifiable group of people with a common 
history and verbal and nonverbal symbol system” (p. 18). Intercultural sensitivity is not natural and training in 
intercultural communication enables people to overcome and transcend traditional ethnocentrism (Bennett, 1993). 
Success in the workplace and in academic efforts is often seriously limited by a lack of cultural adjustment. 

However, for the purposes of this paper we are interested in exploring only the differences between 
different cultural groups in what they value in education. It is accepted that sub-groups within a country may differ 
in significant ways, but when compared to groups of learners from other countries, sub-groups within a country have 
more in common with each other than with outside groups (Helmer & Eddy, 2003). Ramirez and Price-Williams 
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(1974) and Neuliep (2003) have noted that different subcultures within the same country exist in ways that are as 
different as we expect cultural differences in nationalities from persons from other countries. Some of the same 
guidelines for creating face-to-face instruction for diverse classes apply equally well to an international IBL course. 
If your situation meets the criteria described in this paper then we recommend that you employ the design heuristic 
at the end of this discussion. Increasingly in the 21 st Century, academics are suggesting that there is no one best way, 
fixed way, or one-size -fits -all way to teach language or culture (Edge, 2004) so we encourage you to be creative. 

Cultural differences created by language and the various educational and social systems around the world 
produce learners who are educated, trained, and comfortable learning under different conditions (Hofstede, 1986; 
Neuliep, 2003; Freeman and Freeman, 2001; Gunawardena, et ah, 2003; Nieto, 2002, and Bennett 1993). However, 
in North America there is a prevalent expectation that those coming to the US should assimilate into the dominant 
culture and adopt its values. Historically, it has not been the norm in the US to value cultural differences and see 
them as contributing positively to a rich educational experience for all involved. Where this attitude might have 
served the country well in the past to unify immigrants, it is a potential weakness for providers who are trying to 
market American-centric IBL to a global audience without trying to account for their differences in educational 
values and social systems. The time to account for these differences starts with the needs and audience analysis 
phase as designers examine their own underlying cultural assumptions and values they have along with the 
assumptions they make concerning their learners’ profiles and ability gaps. 

Until they take university courses designed from a different cultural orientation, learners may not realize 
the effect the clash of educational values has on their ability to be successful in course. Solano-Flores and Nelson- 
Barber (2000) held that “because culture and society shape mental functioning, individuals have predisposed notions 
of how to respond to questions, solve problems, and so forth.” These predispositions influence the way students 
interpret, respond, and reasons so as children grow, they learn how to think and live within a given language and 
culture. Adult learners have developed definite ideas about what kind of learners they are and what is an acceptable, 
comfortable way to learn from their culture’s perspective (Gunawardena, et al., 2003). Creating a better match of 
course offerings for adult worldwide learners will require some reeducation on the part of both the course designers 
and the learners (Hofstede, 1986; Nieto, 2002; Smith, 2001; Gunawardena, et al., 2003, Echevarria, 2000; Palloff 
and Pratt, 2003; and Freeman and Freeman, 2001). 

3. Technical infrastructure differential 

Although instructional designers and learners in IBL courses will have different cultural backgrounds and 
educational values, naive designers may plan the course in terms of the global reach of the course’s technical 
capabilities to teach in any nation with sufficient infrastructure. They think only of bandwidth, access to email, and 
processor speed. Economic reviews such at the Global Information Technology Report assess each “nation's 
[technical] environment for the development and use of Information and Communication Technologies (ICT); the 
readiness of the community (consumers, business and government); and communities' usage of ICT” (World 
Economic Forum, 2003). Although it is crucial to know the technical ability of a country to receive IBL content, 
they must also see it as a desirable thing to have. It is very ethnocentric to believe that other groups of people see 
things the way Americans do with the same assumptions, values, and core beliefs. It is important to remember that 
technical reports like the Global Information Technology Report do not make any attempt to assess the educational 
openness associated with embracing courses built on different educational values. 

4. Local verses global differential 

It is important to remember that the learner is usually taking an IBL course from a local perspective and is 
using its website under varying circumstances, some of which, the designers are not familiar with (Main, 2002). 

Main goes on to explain that because of the general ease of creating IBL courses with popular authoring tools, the 
general look of courses are “more or less preset and does not take into account the subjective and objective cultural 
issues specific to target cultures.” Simon (1999) found that subjective culture is psychological and deals with 
attitudes. Local context is often valued over global context, and yet there is a rush to embrace more aspects of 
globalization with its dependence on Internet technologies and worldwide connections (Friedman, 2000). It has been 
our experience that LMS vendors frustrated with time and costs associated with assessing the differences between 
local and global perspectives prematurely choose to ignore them in an attempt to follow a cost more effective 
development model. 

5. Learning style differential 

Student’s attitudes are based on the experiences, values, and the different ways of mental programming of a 
culture. Education is value laden, and how learners perceive “Good” instruction is based on what they think and 
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value. What makes one group of learners happy is just as likely not to meet the needs of another group of learners. 
Martinez, et al. (1999) and Bentley (2000) have shown that the same learners who prefer loosely structured flexible 
environments that promote challenging self discovery are unlikely to be comfortable learning in highly structured 
environments that deal with simple solutions and a large amount of strictly guided instruction. In the authors’ 
opinion, the instructional designer and the learner need to share responsibility for knowing what educational values 
they hold. The designer has the responsibility to make the courses educational values explicit in the course 
materials. It is the learner’s responsibility to understand themselves as learners and find out about the context from 
which the course originates. 

It is difficult for non-native speakers to learn higher level thinking and language skills in online courses that 
are not designed to accommodate their thinking and learning styles. Shadbolt (2002) supports the concept of various 
learning styles across cultures and maintains that typical American tell-and-test training materials “would be 
regarded as too authoritarian a style of teaching” in “parts of Europe, particulaily in the UK. . . People here [Europe] 
prefer more of a self-discovery approach, particularly in the soft -skills training” (p. 51-55). Many American training 
products use models that do not fit the varying teaching and learning styles in different cultures. 

6. Reasoning pattern differential 

Thinking patterns in the form of reasoning and approaches to problem solving are valued differently from 
culture to culture. The thinking pattern most prevalent in the dominant culture is usually the most highly valued. 
Depending on the worldview and culture through which learners filter their perceptions, they may perceive the same 
object in different ways according to their culturally dominant thinking pattern. Gunawardena, et al., (2003) wrote 
how a noticeable characteristic of Anglo-Americans’ communication style is direct because they think in a “line”, 
while the Japanese, for example, think in non-linear “dots”. 

A useful analogy is that Anglo-Americans use the “bridge” model of thinking, which is characteristic of 
linear thinking, in that they send ideas explicitly and directly from point A to point B. The meaning found in the 
words themselves is expected to be enough to communicate. On the other hand, the general Japanese “stepping stone 
model” of meandering dots is characteristic of circular thinking and sending ideas indirectly for others to surmise the 
meaning. The indirect or non-verbal cues in the setting, body-language, tone, pauses, silence, and the status of 
individuals are important to communicate the meaning. Just words themselves without their specific context and 
setting are not enough to communicate meaning. 

7. High and low context differential 

In Table 1, Edward Hall (1966, 1976) “compares the cultures of the world on a scale ranging from high- 
context to low-context” (Main, 2002). The high-context, circular thinking model of group-oriented cultures such as 
the Japanese, Chinese, Korean, Latin American, Mediterranean, Middle Eastern, French, and Vietnamese cultures is 
noted in column one. The characteristics of low-context cultures, where the focus is more on individuals and not on 
the group, are listed in column two. Low-context cultures are represented by the United States, Canada, the United 
Kingdom, Germany, Australia, and most of Western Europe, including Scandinavia (Neuliep, 2003; Gundling, 

1999). In education, countries described as low-context offer what is sometimes referred to as Western -style 
education. 

Many high-context international learners have difficulty using online courses prepared in the United States 
because of their limited ability in English as well as their conflicting learning preferences which do not easily 
accommodate to using materials prepared by and for low context culture users (Anglo-American users). Hofstede 
(1986) explains that "academic learning in different industrial countries appeals to different intellectual abilities." 



Table 1 Characteristics of High-Context and Low-Context Cultures 


High-Context Culture 


Low-Context Culture 


Implicit messages 


Explicit messages 


Internalized messages 


Plainly coded messages 


Nonverbal coding 


Verbalized details 


Reserved reactions 


Reactions on the surface 


Distinct in-groups and out-groups 


Flexible in-groups and out-groups 


Strong people bonds 


Fragile people bonds 


High commitment 


Low commitment 


Open and flexible time 


Highly organized time 
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Differences in thinking patterns can lead to misunderstanding in intercultural communication and in 
education because these affect students in the following ways: in how they interact with course content, 
assumptions designers make in designing the course content, and expectations about what courses offer and how to 
successfully complete them. 

8. Social context differential 

The theory of situated cognition describes how learners respond to new information based on the social 
context (Driscoll, 2000; Henning, 2004). High context learners REQUIRE more social context in order to read the 
meaning of the communication and how to respond appropriately. For a continuum of elements used in 
communication situations, see Figure 1 . 



Relationship between Rapport & Context 



Rapport 




Legend: 



• Email 




• Fax & 




Groupware 


w* 


• Voicemail 




• Telephone 


0 


• Video 
Conferencing 

• Person 


Is 




to Person 





Figure 1. Rapport verses context axis adapted from Gundling (2000). 

Taking Email as an example from this chart, we see that it’s a low context medium that requires a high 
rapport between the sender and the receiver to understand each other’s words. Low-context North Americans for 
example, emphasize the information in Emails by focusing on the exact words, prose style, argumentation and line 
of reasoning, and ideas. To North Americans these are often more important than who the people involved are. Just 
the opposite is true for high-context persons because they are looking for non-verbal cues, social standing, and 
situational contexts to know how to respond appropriately. In many Western societies. Email is seen as a quick, easy 
way to communicate, but this ease of using only words to communicate content and meaning can often put members 
of a high-context culture at a disadvantage. Archee (2003) observed, 

... I do not think that 5,000 years of cultural communication patterns can be changed by mere decades of 
Internet usage, and with today’s vastly increased communications opportunities, I believe we will see an 
equivalent increase in the amount of mAcommunication between cultures. . . When we use e-mail, we 
prefer fast turnarounds and quick decisions. These expectations may be totally at odds with those of our 
Asian partners, who may ignore our demanding emails or feel forced to make premature decisions, (p. 40) 

High context learners do not receive much meaning if it is presented in text only and if they are involved in 
a lexical loop without some person-to-person interaction with others. High context learners struggle as newbie’s in 
online environments when the technologies used actually alter the social presence of individuals and offer few clues 
as to the meanings of some conversations and online content. Gundling (1999) asserts that important messages are 
best communicated through high-context means. In cross-cultural settings he recommends that the facilitator 
increase contextual cues. For example, prior to a videoconference written background material, an agenda, a seating 
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chart, and biographical information about the participants could be circulated in writing. Then during the 
videoconference, the facilitator would introduce people, act a gatekeeper to bring everyone into the conversation, 
and define unfamiliar terms and concepts. 

How these eight value differentials relate to IBL 

The most basic ADD1E model for instructional designers stresses the need to know your audience so that 
your instructional intervention is most likely to meet their needs (Dick & Carey, 1996; Seels & Glasgow; 1998; 
Smith & Ragland, 1999). Designing quality IBL for an international audience is a daunting task. If a mismatch 
occurs when students’ sign up for an IBL course that clashes with their cultural perspectives and learning style 
preferences, it likely that this sector of the market was not included in the needs and learner analysis. While it is 
highly recommended that a through audience analysis be conducted, the authors realize the how difficult it is to try 
to accommodate all learners, everywhere and do not recommend that you try to be everything to everyone. The 
assumptions from the audience analysis which shapes the instructional design should be evident to the learner. 

Understanding what the instructional designer or teacher values, and has built into a course, will help other 
learners anticipate their educational experience and choose IBL courses appropriately. We advocate including a 
new element to the instructional process of analysis and recommend not only seeking to know the audience but the 
designer as well. We encourage students to share the responsibility for finding the right course themselves by self- 
selecting classes they feel would be a good match between their educational values and those of the instructional 
designer using the eight educational value differentials. 

A certain degree of “readiness” is needed to be able to successfully take IBL classes. There are many 
survey forms such as Strategies for Success: Study Skills for Online Learners, currently available that review time 
management, study skills, test taking, and motivation to determine how well suited a learner is for IBL (Alamo 
Community College District, 2002). IBL courses are offered in a location (in “space” and in the mind) that takes 
some adjusting to for those accustomed to face-to-face courses, no matter which culture they come from. The 
Internet exists on servers, wires, protocols, connections, and browsers, but it also exists in the minds of the people 
who use it, perceive it, and build representations of it in their minds (Bruce, 2002, p. 158). 

People relate to the Internet through how it intersects with their lives, uses, applications, and contexts. 

Bruce (2002) explains that “Out of these doings, people build individual constructs of the Internet” and in the form 
of knowledge structures that allow them to interpret and make sense of things (p. 158). Gaps occur in the continuum 
between actually using the Internet, IBL courses offered, and the individual user’s experience. Users can reduce the 
gap with each experience in an IBL course as they incrementally transform their perceptions and abilities with each 
successive using. In the initial stages, however, we believe their first attempts at IBL courses will be more 
successful if they find a course that offers some options that match their culturally based educational values. 

Bentley & Tinney (manuscript under review) found that students with a non-US educational background 
have statistically significant different preferences for how they want to interact with content than students with a US 
educational background. They go on to say that “it might appear to be common sense that cultural differences would 
affect how students learn, but understanding the nuances of those differences and accounting for them in the 
structure of the course is challenging” (p.l). 



Recommendations 

IBL designers, instructors, and students must be aware of the potential conflict in teaching and learning 
contexts. Reed (2002) concludes that “To bridge the gap that occurs in cross-cultural learning contexts, Hofstede 
(1986) proposes two possible solutions; (1) To teach the teacher how to teach, and/or (2) to teach the learner how to 
learn.” 

So far we have discussed how designers should prepare instructional materials, strategies, processes, and 
course components that are adapted to make learning better for learners’ cultural orientation as well as how course 
catalog descriptions or career counselors could be more explicit in matching up users to these types of programs and 
courses. Now we present the following checklist from the research to help aid this process. The checklist is 
primarily categorized into two parts; recommendations for teacher/instructional designers and recommendations for 
students. 

Six Recommendations for Low-Context (American) Instructional Designers: 

1. Explicitly describe the educational values embedded in your course design and in your examples and 

strategies. Include these values in both the syllabus and course description to alert potential students of the 
course orientation. 
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2. Offer optional scaffolding elements to help learners be successful such as mentors, a pre -course orientation, 
and practice in prerequisite skills. 

3. Consider the knowledge and skill level of English required to use the course. When you know you will 
have both native and non-native speakers, be sure to use simple sentence structures. 

4. Avoid slang, colloquialisms, and local humor when possible or explain your intent clearly in the next 
section so they can understand what you intended. 

5. Before any real-time activity, make topic information available ahead of time for students to review in 
order that they may have time to use a dictionary to define new terms, consult with others, and find suitable 
words to express their contributions (Freeman & Freeman, 2001; Smith, 2001). 

6. For IBL courses intended for collectivist societies (high context cultures). Main (2002), Rao (2002) 
suggests that materials should be designed along these guidelines: 

• Place little emphasis on personal achievement; 

• Define success in terms of sociopolitical, rather than individual, goals; 

• Promote group solidarity rather than individual self-interest; 

• Be written in an indirect, impersonal style; 

• Emphasize tradition and history. 

Eight Recommendations Designers Should Make to Their High-Context Students: 

Students from high context cultures are accustomed to a systematic, step-by-step, highly disciplined approach to 
teaching. Hopefully, these suggestions can help high-context learners as they shop for online courses to quickly 
adapt to a low-context IBL learning environments. 

1. Be less dependent on a highly detailed syllabus 

2. Dispel old beliefs about how effective teaching should be taught 

3. Embrace new learning habits and adapt to them, as in an adventure 

4. Do you have an open mind to try some new things? Are you ready to be stretched mentally? Socially? 
Culturally? Technologically? 

5. Do more to figure things out yourself 

6. Join study groups and social groups 

7. Seek ESL help 

8. Talk to the instructor concerning accommodations that can be reasonably made to fit the course to your 
style or ability level. If no reasonable accommodations can be made and you still feel uncomfortable with 
the mismatch, then drop the class. 



Conclusion 

The eight educational value differentials or factors which make a distinctive difference in how the learner 
perceives quality in instruction are language, culture, technical infrastructure, local/global perspective, learning 
styles, reasoning patterns, and social context. 

In designing IBL instruction one should take into account that users may well come from various cultures; 
therefore, the content should be designed as culturally neutral as possible. If instructional designers and students 
will follow our recommendations to discover their educational values and make them explicit, we believe that much 
of the stress and frustration surrounding the mismatch between student educational values and educational values 
embedded in the course by the teacher can be resolved. We should follow Daniel and Macintosh’s (2003) 
recommendation to “be watchful that [IBL] solutions do not entrench the digital divide, or even worse widen it (p. 
822). We should also be a particularly sensitive to the cultural relevance of imposing past successes of the 
industrialized world” into other contexts. 

We should be doing all we can to understand the audience for IBL and what they value. As we have shown, 
how we determine “good” instruction is based on what educational values we hold. Understanding where our 
educational values come from and how they might differ across cultures is important as the internet-based learning 
(IBL) market becomes increasingly global. If we will do so, learners can then choose courses that match their 
educational values for a more comfortable learning experience or know that choosing classes which do not match 
their educational values will require that they learn in new ways. 
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E 2 ML: A VISUAL INSTRUCTIONAL DESIGN LANGUAGE 



Luca Botturi 
University of Lugano 

Instructional Design from Craftsmanship To Production 

The advent of technologies has changed our very idea of what a course is (Bates & Poole, 2003). 

Instructors in Higher Education are now daily supported by instructional designers or educational technology 
experts that provide advice for integrating Web-based activities, videoconference sessions, high-quality digital 
media presentations, etc. in their teaching activities. The process of designing courses has grown a more and more 
structured and interdisciplinary process (Szabo, 2002), one that is too complex for a lone-ranger professor to cope 
with (Bates, 1999). In some respects, teaching is thus developing from craftsmanship to a large scale production 
process (Cantoni & Di Bias, 2002), in which communication has become a critical variable. 

A fairly recent research trend in the field of educational technology is the development of visual 
instructional design languages. This paper is a sort of tutorial aiming to introduce one of these new professional 
tools for designers: E 2 ML- Educational Environment Modeling Language. 

In order to explain the relevance of I?ML, the first section is devoted to the identification of some features 
and issues concerning the Instructional Design process through the analysis of the literature. The second section 
introduces some relevant literature, among which the foundational work by Gibbons, and two other visual design 
languages. E 2 ML is presented in the third section through a detailed example, while additional references concerning 
other studies about the language are provided in section four. The conclusion presents a summary along with 
indications for further work. 



Communication & Instructional Design 

The daily work of the instructor and of the instructional designer in any tertiary educational organization is 
mostly interdisciplinary teamwork (Greer, 1991). The profiles in the team depend on the specific context even if, 
generally speaking, a team should involve “(■■•) any combination of subject experts or faculty, project manager, 
instructional designer, graphic designer, computer interface designer, desktop editor, Internet specialist, and media 
producer, depending on the design of the project” (Bates, 1999, p. 70; see also Achtemeier, Morris & Finnegan, 
2003). 

Each of these professionals makes use of a specific technical language, so that misunderstanding can easily 
endanger successful development. In fact, we cannot give for granted that a Literature professor with educational 
background understands the word ‘‘active learning” or ‘‘creative discussion” in the same way as a Web programmer 
does. Nevertheless, the person in charge of creating the artifacts that should foster the learning activity is the latter. 
The quality of the educational experience heavily depends on the communication between the two. Such a scenario 
grows more complex when eLearning programs require collaboration between different institutions or organizations. 
These problems clearly call for the definition of a lingua franca among the different profiles involved in 
instructional design. 

Other issues are at stake too. How can the final learning activity maintain its overall consistency? How to 
seamlessly merge the contributions of all profiles into one final ‘‘product”? The overall complexity of the design of 
instruction can be managed by assigning specific tasks to several specialists and by organizing the production 
process into phases, following a project management approach (Bates, 1999; Greer, 1992). Several models of 
Instructional Design have been proposed describing the main phases a well-structured project should undergo, 
summarized by the basic steps of the ADDIE model, which reflects much of the practice as designers describe it 
(Rosenberg, Coscarelli & Hutchinson, 1999): Analyze, Design, Develop, Implement, and Evaluate. 

The management of the design and development process is based on what Greer (1992) and Bates (1999) 
call a blueprint - usually a written text in natural language. Is there a way to produce a more standard and synthetic 
description of the instruction? In particular all Instructional Design models include the evaluation and revision 
phases at the end of the design process (e.g. Dick & Carey, 1996), and some of them suggest a constant tryout and 
revision process (e.g. Greer, 1992). While a thorough control of quality is necessary, both solutions are costly, as 
they take place after the production has started, even if in some cases they only involve prototypes. Is it possible to 
support at least a partial quality check at design time? 

Finally, after a course has been developed, usually the only documentation available are the actual learning 
materials. This raises some issues in the case a redesign or adaptation process is required for reuse, especially in the 
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case the original designer is not available. Is it possible to produce a documentation that can guide the reuse and 
adaptation of the instruction? 



A Language For Instructional Designers 

In a recently published article. Waters & Gibbons (2004) provide an interesting approach to instructional 
design communication. Drawing examples from Dance, Chemic and Architecture, the two authors point out that 
almost any creative and technological field has developed one or more notation systems and design languages. 
According to their definitions, (a) a design language is a personal and abstract set of concepts that a designer can use 
for creating design structures; while (b) a notation system is a tool for providing imperfect but visible and public 
expression to design structures. In other words, an architect creates a new and original building thanks to a set of 
aesthetical, compositional and technical concepts, and than expresses her ideas through a set of drawings that allow 
her to share the project with other people. 

As Waters and Gibbons emphasize, there is a tight relationship between design languages and notation 
systems, as between our thought and our mother tongue. “As designers improve and extend their personal design 
languages, this in turn calls for extensions and improvements to the notation system. The notation system then is 
capable of expressing more interesting and complex designs and easily leads to innovation.” (Waters & Gibbons, 
2004; p. 59). This looping relationship is summarized in Figure 1. 



gfces new :efrts to 




Design Notation 

Language System 




gum new terms to 



Figure 1 - Cycle of improvement (taken from Waters & Gibbons, 2004, p. 59) 

Coming to the field of Instructional Technology, the authors report that instructional designers use 
idiosyncratic and “personal” design languages and notation systems, while no complete blueprint language exists. 
The work of Horn (1974) and Merrill (1983) represent a step toward the development of a language, but their 
contributions mainly focus on a particular layer of a design - namely Horn on content structures, and Merrill on 
strategy structures. Eckel (1993) has also proposed an instructional language centered on interaction design. 

Gibbons & Brewer (2004) identify at least three benefits that ID would get from a shared notation system: 
(a) remembering designs; (b) having a virtual problem-solving workspace, for trying out new solutions; and (c) 
having a lab for sharpening concepts and merging them. To these I add the improvements of a shared language for 
discussion, through which different design traditions and schools could meet. 

The literature research conducted during the development of E 2 ML also revealed the lack of a visual design 
language for Instructional Design. Morimoto (2003), working on notation systems for lesson plans, also reports the 
lacking of such a design tool. 

Recently, the introduction of Learning Technology standards has brought to the developments of languages 
that could serve as an interface between the humanistic world of instructors and the technical one of IMS and 
SCORM. The works by Lischka (see e.g. Lischka & Karagiannis, 2004; Bajnai & Lischka, 2004) with eduWeaver 
and Derntl with the Person-Centered e-Leaming patterns (Derntl & Mangier, 2004; Derntl & Motschnig-Pitrik, 

2004) are a positive advancement in such direction. 

E 2 ML Language Definition 

E 2 ML - Educational Environment Modelling Language is a visual language for instructional design. Its 
general approach is visualization, and it is targeted to instructors and instructional designers. 

The main issue E 2 ML is concerned with corresponds to what Greer (1992) and Reigeluth (1983) called the 
development of a blueprint: a representation of the instruction that all stakeholders, designers, developers and 
instructors can see, understand in a similar way and, hopefully, agree upon. According to the categories proposed 
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by Gibbons & Brewer (2004), E?ML is a design language with a very limited number of basic concepts, coupled 
with a visual notation system. 

This section introduces the language through the example of a two -day intensive course in Effective 
Mediated Communication (EMC) delivered by the author to 20 commercial managers from an international private 
Swiss-based healthcare firm in 2004. 



Document Sets 

An E 2 ML blueprint consists of three sets of documents. Each of them provides support for specific design 
tasks. The three sets are: 

1. Goal Definition, i.e., a declaration of the educational goals. This is composed by two documents: the goal 
statement and the goal mapping. 

2. Action Diagrams, i.e., the description of the single learning and support activities designed for the instruction. 

3. Overview Diagrams, i.e., two different overviews of the whole design, the dependencies diagram and the 
activity flow. 



The documents are described in the following sub-sections in their standard form. As any real design 
process and any real instructional situation have their own unique features, they can be adapted (simplified or 
detailed) to the needs of the specific context or design team. They are produced at different moments in the design 
process, and do not have a tight correspondence with specific phases. The order of presentation in this paper, which 
does not follow the numbering of document sets, was selected for its suitability to the case study, and should not be 
understood as an indication of method. The elements of E 2 ML can be implemented flexibly, in a sequence tailored 
to the needs of each project. 



Goal Definition (Document Set 1) 

Expressing learning goals means creating a compass for design, and is important for different reasons: 
selecting what to teach, how to teach it, what to evaluate and how to make the whole instruction consistent 
(Anderson & Krathwohl, 2001; Gronlund, 1995; Yelon, 1991). The most widely known models for learning goals 
(or instructional objectives) are those by Bloom (1956; 1964), Gagne, Briggs & Wager (1992) and Merrill (1983). 
One central issue for the development of relevant and consistent goals is collaboration with Subject-Matter Experts 
(SME) and negotiation with stakeholders. A second issue is creating a common understanding in the whole team 
about the goals as primary requirements for the project. The first E 2 ML document set aims to provide a support for 
these two communication processes. 

Goal Statement Table. The goal statement describes learning goals (possibly sub -goals and objectives) according 
to an extensible set of parameters, the core being an identifier tag, the goal statement, the target, the main 
stakeholders, the instructional approach and an importance score. The list can be extended according to the needs of 
each project. The EMC course had the following goals (Table 1): 
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| GOAL STATEMENT 


TAG 


STATEMENT 


TARGET 


STAKEHOLDER 


APPROACH 


IMPORTANCE 


A1 


Recognize critical success factors in 
communication 


All 


Head 




5 


A2 


Analyze successful and unsuccessful 
communications 


All 


Head 


Case studies and 
discussion 


4 


A3 


Recognize differences between direct and 
mediated communication settings 


All 


Head 




5 


B1 


Recall key concepts of communication 


All 


Head 


Critical discussion 
on movie clips 


4 


Cl 


Perform effective videoconferencing 


All 


Head 




2 


C2 


Perform effective audioconferencing 


All 


Head 


Guidelines, 


3 


C3 


Write effective emails 


All 


Head + Mktg. 
Director 


examples and 
exercises 


5 


C4 


Deliver effective presentations 


All 


Head + Mktg, 
Director 




5 


C5 


Effectively integrate corporate Web sites into 
communication 


All 


Mktg. Director 




3 



Table 1. Goal statement for EMC 



The goal statement table is an orderly summary of the goals of the instruction. Yet often the verbal 
expression of goals can be ambiguous or unclear, especially in a multidisciplinary environment. For this reason the 
goal mapping, presented below, can be a powerful comple ment in the goal definition process. 

Goal Mapping. In order to enhance communication, learning goals can be visually expressed by mapping them on a 
visual grid or representation, such as Merrill’s Content-Performance Matrix (1983), the revised Bloom’s taronomy 
(Anderson & Krathwohl, 2001), or the QUAIL model (Botturi, 2003 a; Botturi, 2004 a). According to Anderson & 
Krathwohl (2001), trying to classify a goal lets all implicit understandings emerge, and is a chance to align the 
whole team. Two points deserve great care: first of all, the representation device should be consistent with the kind 
of goals addressed (cognitive, psychomotor, affective, etc...); secondly, the designer should be familiar with the 
representation and be conscious (if not share) its underlying implications for learning. This is why E 2 ML simply 
suggests the use of visuals, leaving the choice to the designer. 

The goals for the EMC course are visualized in Figure 2 using the Quail model. 



ACTION 



0MMI7TMEN 



REFLECTION 



CONCEPT 

INSIGHT 




ENGUIRX 



EXPERIENCE 



REMEMBER 



USE 



RIND 
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Figure 2. Goal visualization for the EMC course with the QUAIL model 
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Overview Diagrams (Document Set 3) 

Document set 3 contains diagrams that provide “the big picture”, a synthetic view of the whole instruction. 
They are probably the most interesting for practitioners. Overview diagrams serve as reference for planning the 
development process and as organizers of other documents and project deliverables, and are a powerful 
communication tool both for intra-team communication and for negotiation with clients and external partners. These 
diagrams are conceived for a continuous development throughout the design process, thus becoming a sort of 
interactive shared map of the instruction. 

Dependencies Diagram. Each educational environment has a deep structure that connects its activities and creates 
one meaningful whole. These relationships are not necessarily mimicked in the streamlined disposition of the 
activities on the schedule. 

Activities (or actions - a definition is provided below) are represented by boxes. The relationships 
supported by E 2 ML are: (a) Learning prerequisite: the first action provides a learning outcome that is the 
prerequisite for the second action (e.g., a lecture provides concepts for the following analysis work); (b) Product: the 
first action produces some artefact that is required as input for the second action (e.g., a group-work activity 
produces a presentation which is shown during the following class discussion). Product arrows may be tagged with 
an indicator of the product (e.g., report); and (c) Aggregation: an activity is part of another activity (it is a sub- 
activity). For improving legibility, actions can be grouped into trails, or logical groups of actions, e.g., all lectures, 
or all the actions that form a specific activity in a course, etc. 

The EMC course includes three main groups of activities (grouped with trails): some introductory lectures, 
in-depth practice-oriented sessions about specific media, and exercises. Before the course, participants are asked to 
critically scan the last emails they received and to identify the “most and least communicative ones”, trying to figure 
out why. The resulting diagram, which actually contains more information about specific dependencies, is 
represented in Figure 3. Dot-end arrows are learning pre -requirements, while simple arrows are product 
relationships (no aggregations are shown in this case). 




Figure 3. EMC course dependencies diagram. 



During design and development, the dependencies diagram can be useful to identify cross-unit connections, 
and to provide developers with an idea of the “big picture” that the instructor has in mind. Moreover, imagine that 
during the course a session has a poor outcome for contingent reasons (e.g. a technical failure in the AV equipment): 
the dependencies diagram provides a support to identify (a) possible waterfall effects on other activities, and (b) 
potential activities in which to propose a remedy. 

The EMC course activity flow (Figure 4) exploits an hourly grid, and includes the same activities displayed 
in the dependencies diagram plus two introductions and two course evaluation discussions - they were not included 
in the dependencies diagram as they do not have tight connections with other activities. While the dependencies 
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diagram was built following the inner relationships of the subject matter, the flow shows some adaptation to time 
constraints and a distribution of workload along with a continuous change in the type of activity. The 
ARGUMENTATION theory part for instance, which is not a pre -requirement for any other activity, was placed in a 
different moment than the other theory parts, which are grouped in Day 1. 



| PRE WORK ON EMAIL 



INTRO DAY1 



CONCEPTS 

| 1 0.20 - BREAK 

INFO QUALITY 



PRAGMATICS 
COMM TOOLS 

I ' 1 2.1 0 - LUNCH 



| EMAIL COMMUNICATION 



I ' 14.10 - BREAK 

PUBLIC SPEECH 



PUBLIC SPEECH 1 



EVALUATION DAY 1 



DAY 1 

9.00 

i o.o’o 
11.00 
12.00 

13.00 

14.00 

15.00 

16.00 
17.00 



PUBLIC SPEECH 2 



ARGUMENTATION 



WEB COMMUNICATION 



AUDIOCONFERENCING 



VIDEOCONFERENCING 



• 10.00 -BREAK 



- 12.30 -LUNCH 



EVALUATION DAY 2 



DAY 2 

9.00 

10.00 

11.00 

12.00 

13.00 

14.00 

15.00 

16.00 
17.00 



Figure 4. EMC course activity flow 

The Activity flow can help to keep track of the chronological sequence of activities and to manage complex 
situations with structured group work or multiple student options. 

Action Diagrams (Document Set 2) 

As already mentioned, E?ML only includes a very limited set of design concepts. The main one is that of 
action : a learning environment is a context in which events happen (cfr. Gagne, Briggs & Wager, 1992). 

For E?ML the action is the performance of a set of acts with a unity of purpose by defined acting subjects. 
Unity of purpose means that the action is aiming at one thing, e.g., producing a report, completing an exercise, 
achieving the understanding of a concept, etc. The acting subjects can be a single learner along with the tutor, a 
whole class with the instructor, a tutor alone, etc. An action can be split in several sub-actions according to the time 
and/or space unity criterion in the specific setting (a single lecture or a videoconference). This second distinction 
(time/space) should of course match with the previous one (goal/subject). 

The general schema for the representation of an action is presented in Figure 5. The upper part of the 
diagram contains the proper identification for the action, i.e., its identifier tag, name, type (learning or support) and 
the involved roles (the acting subject). The middle-left area describes the initial state, i.e., the necessary and 
sufficient conditions for learning to be achieved, or for the performance to be successfully completed. The middle - 
right area describes the (desired) final state after the action performance. Finally, the lower part of the diagram 
contains a description of the action performance, including locations and tools. The squares hanging on the right- 
hand side are references to the learning goals as defined in the goal statement, thus providing a tight connection 
between goals and activities. For a detailed description of all fields please refer to (Botturi, 2003 b) 
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ACTION NAME 


TAG 


ROLES 


TYPE 


PREREQUIREMENTS 


EXPECTEDOUTCOMES 


PRECONDITIONS 


SIDE-EFFECTS 


INPUT 


OUTPUT 


PROCEDURES +DURATI0N 


LOCATIONS 


TOOLS 



Gx 



Figure5. Action diagram schema 

As an example. Figure 6 shows the first activity in the EMC flow - namely the email observation pre-work . 



PRE WORK ON EMAIL 


PRE WORK 


All participants individually 


LEARNING 


[no pre-requirements] 


Develop a critical attitude 
toward daily email 
communication. 

Arise some questions about the 
meaning of effective 


Daily email use 


[no side-effect] 


Invitation to do the job by the 
head 


Good and bad emails with short 
explanations (to be sent to the 
instructor) 


Personal work: one week prior to the course, each participant is 
asked by the head to do this activity. They should take 10 minutes 
every day to review their email exchanges and to select the best 
and worst emails they wrote and received. They then write a 
summary, including the email bodies and send it to the instructor. 


Anywhere 


[no support] 



Figure 6. Email pre -work in the EMC course 

Action diagrams provide a synthetic yet detailed description of the very bricks of the instruction: teaching 
and learning activities. Although time-consuming in production, these diagrams can be taken as a complete 
documentation of the instruction for archival and reuse. Once more E?ML supports the description of the instruction, 
and does not directly provide design methods. By doing this, it offers tools for not overlooking details, such as 
locations or tools required for some activity, and for seeing the project in a structured and synthetic way. 

Who Uses E2ML? 

E 2 ML was developed for instructional designers, and every effort was made in order to make it usable, 
understandable and practical to them. In the same way they develop their own jargon - specifying terms as template 
or blueprint, or creating expressions as round disclosure - designers should also feel free to take E 2 ML or any of its 
parts, and extend it, adapt it and make it suitable to their problems. E 2 ML can be also used only partially, without 
exploiting all its features or using them only for some activities. 

Novice designers could use E 2 ML as a language for practicing design. From this perspective, having a 
language means having a possibility to focus on design itself without slipping away to development- which is 
particularly easy if learning materials are the only tangible product of the whole process. 

Should or could E 2 ML visualizations be used with students? E 2 ML diagrams are not conceived for them in 
the same way technical blueprints for a two -floor house are not the best support for letting the senior couple that 
bought it dream about their retirement. Nevertheless, a visualization of the flow of specific activities is proved to 
enhance student performance in particular settings, such as problem-based learning (Santoro, Borges & Santos, 
2003). Diagrams could also be used for negotiating some steps in the instruction, and to improve the critical 
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comprehension of the learning process. In order to make them more effective, the style of diagrams should be 
rearranged and made more appealing. 

Although the analysis of a single case might not have made it clear enough, E?ML does not have a unique 
access point, and does not impose to start e.g. from goals. It is a language, and as such it can be used with different 
strategies that could be taken from other Instructional Design models. 

Other References For E 2 ML 

A more detailed introduction to E 2 ML can be found in (Botturi, 2003 a). Other publications (Belfer & 
Botturi, 2004; Belfer & Botturi, 2003) explore the use of the language for the definition of pedagogical patterns, i.e. 
reusable gist of solutions to recurrent instructional problems. The complete description of the language, along with a 
critical comparison to other ID models and with learning technologies standards can be found in (Botturi, 2003 b). 

A first evaluation of the perception of usefulness of E 2 ML is reported in (Botturi, 2004 b): its results clearly 
indicate that instructional designers see a visual language as a potentially useful tool for their practice, given that it 
is simple, flexible and with a plain learning curve. According the their perception, E 2 ML is mostly useful for 
keeping the overall consistency of a course, and in particular to discuss the consistency of goals and instructional 
activities with the instructors or course authors, as “they usually discuss the goals and then forget them in the actual 
planning”. Designers also think that E 2 ML is useful to blueprint a course, as it “works well in organizing people's 
thinking”, and “may speed up collaboration”, also allowing a greater detail than textual blueprints. Finally, it helps 
to “make the evaluation more evident”, identifying activities in which the achievement of specific goals is assessed. 

Conclusions 

The challenges of the last decades to Higher Education brought to a shift in the idea of course, which 
resulted into a more complex professional environment for instructional designers. This is particularly true from the 
point of view of communication: a design team is by nature interdisciplinary and in interaction with stakeholders, 
external partners and other design teams. E 2 ML was conceived as a visual language that can smoothen and enhance 
project communication. The structure of the language, articulated in three document sets, was presented through one 
example. Its main idea is modelling the instruction as a set of interrelated actions, aimed at goals and performed by 
actors with specific roles. 

The references provided can bring further information and research reports on this language to the most 
interested readers. As a practitioner, I still think that the best way to learn a language and to assess the benefits of a 
tool is trying it out hands-on, playing around a little bit and integrating it in my personal way of designing while 
sharing it with colleagues, thus creating a mutual understanding. 

The main assumption behind E 2 ML is that a special visual language may enhance communication, and 
enhanced communication may improve design; improved design may increase the quality of educational programs. 
In Italy, where I come from, five centuries ago, travelers to the New World brought back a new vegetable, before 
unknown in Europe; the tomato (actually, the first ones that arrived in Europe were golden, and not red, and we still 
call them pomodoro - golden fruit). That novelty, in the hand of experienced and creative cooks, contributed to the 
growth of our culinary tradition, which is now famous all over the world with pasta and pizza (actually, we do have 
much more than that!). I hope that this language, in the hand of experienced and creative designers, might contribute 
to enhance the quality of education and learning. 
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Abstract 

A report on the efforts made to describe the range of human-computer interaction skills necessary to 
complete a program of study in Instructional Design Technology. Educators responsible for instructional media 
production courses have not yet articulated which among the wide range of possible interactions students must 
master for instructional media production purposes. A hierarchy of human-computer interactions is introduced. The 
method and results of a preliminary study of 12 student projects are described. 

Introduction 

This is a report on preliminary efforts made to determine which among the multitude of human-computer 
interactions should be emphasized as part of course of study in Instructional Design Technology. The purpose of this 
report is to inform those instructors and policy makers responsible for the design and delivery of courses in 
instructional media production as part of a program of study in Instructional Design and Technology. The authors 
sought to determine which aspects of human-computer interaction and design are critically important to include in a 
course of study that prepares one to take on the responsibilities of a professional instructional designer. There is 
currently very little written or reported on this topic. The authors suspect that many, if not all, programs of study in 
Instructional Design Technology rely solely on the judgment of individual faculty members to decide what types of 
human-computer interactions should be mastered by students within their individual programs. 

A review of graduate programs in Educational Technology, Instructional Design, Instructional Systems, or 
Instructional Technology offered by accredited post-secondary institutions in the United States (including Florida 
State University, Indiana University, Pennsylvania State University, San Diego State University, Syracuse 
University, East Carolina University, California State University, Fullerton, University of Colorado at Denver and 
University of Georgia), referred to collectively as Instructional Design/Technology (IDT) programs hereafter, 
revealed that all contained as part of their program of study at least one instructional media production course 
requiring the use of some type of computer-based authoring program (e.g. Director, Flash, Toolbook, HyperStudio, 
Authorware). A simultaneously conducted review of jobs requiring a degree in IDT (jobs posted online at AECT, 

The Chronicle of Higher Education and Jobsearchsite.com) indicates that the majority of these positions specify the 
need for skill in computer based training (CBT), thus justifying the program requirements of the schools reviewed. 

Every IDT program requires the development or demonstration of some skill with authoring software in 
order to create at least a working model of a computer-based, instructional interaction, and most jobs advertised call 
for experience in this area. None of the IDT programs of study reviewed currently require that students demonstrate 
competence in computer programming (using languages such as C++, or Java) or network administration, nor do the 
vast majority of IDT positions advertised require these advanced computing skills. It can therefore be inferred that 
the discipline of IDT does not require advanced level skills with computing machinery (e.g. “hardcore” 
programming or network administration certification), but it does call for some ability to create human-computer 
interactions at a level that is more sophisticated than a common PowerPoint presentation. The standard 
competencies articulated by the International Board of Standards for Training, Performance and Instruction 
(IBSTPI) (Richey, Fields, and Foxon, 2001) reflect this as well. The essential competencies that are specifically 
addressed by requiring students to work with computer-based authoring programs include: 

3. Update and improve one’s knowledge, skills and attitudes pertaining to instructional design and 
related fields. 

b. Acquire and apply new technology skills to instructional design practice. 

11. Analyze the characteristics of existing and emerging technologies and their use in an instructional 
environment. 
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a. Specify the capabilities of existing and emerging technologies to enhance motivation, 
visualization, interaction, simulation, and individualization, 
c. Assess the benefits of existing and emerging technologies. 

It appears that IDT students require more than the ability to control human-computer interactions such as 
conducting slideshow-type presentations but less than the ability to write computer software using a programming 
language. Knowing that these are the far ends of the spectrum, the question becomes, “What lies within the range 
between these extremes?” Knowing the answer to this would pave that way for an answer to the question, “What 
human-computer interactions should professors of Instructional Design/Technology (IDT) expect students to be able 
to control?” In other words, regardless of the authoring tool required or recommended, what elements of human- 
computer interaction must an IDT professional have control over in order to do his/her job well? 

Those responsible for teaching instructional media production courses within IDT programs have not yet as 
a group articulated which among the wide range of possible interactions are necessary to command in order to 
effectively produce instructional media. Furthermore, lacking a description of the range of possible interactions, one 
cannot describe what is considered beginner, intermediate or advanced control of these tools. In order to ensure 
consistency and quality within programs of study in IDT, a more complete articulation of the HCI capabilities 
important to members of the field is needed. 

Defining Human-Computer Interaction (HCI) 

One problem encountered when attempting to describe the interactions necessary to complete an 
instructional design project is the nebulous nature of HCI as a field of study. According to the Association for 
Computing Machinery’s special interest group on Computer-Human Interaction (ACM SIGCHI, 1996): 

“There is currently no agreed upon definition of the range of topics which form the area of human- 
computer interaction. Yet we need a characterization of the field if we are to derive and develop 
educational materials for it. Therefore we offer a working definition that at least permits us to get down to 
the practical work of deciding what is to be taught: 

Human-computer interaction is a discipline concerned with the design, evaluation and 
implementation of interactive computing systems for human use and with the study of major phenomena 
surrounding them. 

From a computer science perspective, the focus is on interaction and specifically on interaction 
between one or more humans and one or more computational machines. The classical situation that comes 
to mind is a person using an interactive graphics program on a workstation. But it is clear that varying what 
is meant by interaction, human, and machine leads to a rich space of possible topics...” 

For purposes of current needs within the field of IDT, HCI study can be limited to the interaction between 
one or more humans and one or more standard computers. A standard computer is defined as a computing machine 
that accepts input via the popular devices of mouse, keyboard, or touch-screen/stylus, and outputs information via 
visual display (monitor or LCD) and audio (speakers or earphones). 

Method 

Very little has been written on determining which human-computer interactions should be emphasized for 
purposes of instructional media production and HCI in general is a nebulous area. Even with limiting our attention to 
standard computer hardware, we found ourselves needing to experiment with ways of articulating HCI in general. 

We also needed to find a way to compare HCI elements in general to those elements that are most often used for 
student-created instructional media projects. 

Step One: Articulating the range of human-computer interactions. 

To begin to address the questions posed, the authors discussed a series of HCI interactions, describing the 
type of interaction and suggesting the level of computing sophistication necessary to complete the interaction. The 
two modes, “active” (or “command”) and “passive” (or “reception”) are divided into four hierarchical categories: 
“basic,” “simple,” “advanced” and “programming.” One of the authors devised a graphic representation of the 
possible levels of human-computer interaction (Figure 1). 
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Programming Reception 



Programming Interactions 



Command Interactions 




An intermediate input device 
suggests that it requires some 
training or orientation - that tire 
device is not a 'natural 
affordance ' I'm thinking a 
joystick would qualify as a 
natural affordance-type of device, 
hut a keyboard would be an 
intermediate device because it 
requires orienting oneself to tire 
keys 



Basic Command Interactions 



The idea of dual modes (passive and active) and four general levels (basic, simple, command and 
programming) is a first attempt to articulate the variety of human-computer interactions that are possible. They are a 
“talking point” through which the authors gained a common language between themselves about which HCI 
interactions are necessary to command for effective instructional media production. 

Perhaps the most useful aspect of the modes and levels idea is that it begins to form the foundation for what 
may be considered beginner, intermediate ox advanced control of human-computer interactions. 

Step Two: Examining the human-computer interactions applied to course projects. 

The next step in answering the question, “What human-computer interactions should professors of 
Instructional Design/Technology (IDT) expect students to be able to control?,” was to examine a variety of projects 
completed by students in instructional media production courses. The authors examined twelve projects created for a 
variety of courses (all taught by the authors). The projects were computer-based instructional media produced within 
the last three years. The projects were created using computer-based “authoring tools,” including Authorware, 
Director and Flash. A chart was created using the two broad categories mechanics and design. Within each of the 
two broad categories a series of sub-categories and the specific examples within these sub-categories were described 
(see Table 1). Each of the projects was then scrutinized to determine if they contained examples of the human- 
computer interactions listed. It is important to add that we use the word “interactions” because it is part of the 
common parlance of those who study HCI; we actually use it as a synonym for the phrase, design intention or design 
consideration. In many cases, there are no specific interactions that can be observed. For example, “Text” can be 
observed (text is either present or it is not) but, strictly speaking, it is not an interaction. 
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The authors then selected 12 examples of student created, computer-based instructional media developed 
for courses that emphasize instructional media production. Each was examined to determine which specific 
examples of the human-computer interactions listed in Figure 2 could be directly observed in the student project 
(during this process, a few specific human-computer interactions were added to the initial list because examination 
of the project spurred further consideration between the authors). 

A simple tally of the instances when specific human-computer interactions were observed revealed that a 
few interactions were ubiquitous (occurring in all projects) and some were common (occurring 9 times or more), 
while others could be observed less frequently, and a few were not observed at all. 



Sub-Categories 



Still Graphics 




|Specific, Observable Instances 



Number of 
Instances 



Images 


12 


Text 


12 




Animated Text 



Animated Images 



Animated Buttons (rollovers) 



Video Sequence 



Screen Changes (transitions) 




Overt Buttons ('standard' buttons) 


10 


Images that Act as Buttons 


8 



Input Fields 


6 


Output Fields 


8 




Math Functions 



Writing function 



Review Function 



If-Then Situations 



As Part of Video Sequence 



Sequenced with Animated Text or Graphics 



Sound effects 



Narration 



Background Music 



Linear Navigation 



Non-Linear Navigation 



Menus 



"Next" and "Back" Buttons 



Local Variables 



Global Variables 



Basic Math Functions 



Writing function 



Review function 
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Triggering a dialogue box ("you didn't fill in 
your name") 


5 






Allowing a user to continue (if the user answers 
correctly, they may go on) 


7 




Testing 










Multiple Choice Questions 


6 






Feedback (correct and wrong) Per Question 


9 






Feedback for Completed Test (Test Score) 


5 






Short- answer 


5 






Drag/Drop 


2 




Graphic Design 










Consistent "Look and Feel" 


9 






Age Appropriate Font Size 


10 






Highlight keywords 


2 






Used for stories 


3 




Writing 










Grammar, Spelling, Puncuation (high quality) 


6 






Well-Crafted Text Content (well-written text) 


8 



Table 1: The categories, sub-categories and specific human-computer interactions looked for in the student projects 
examined, along with the number of times each instance was observed. 

Listing these instances in order of the number of times they were observed (Table 2) begins to reveal a 
sense of which human-computer interactions are critically important in a student instructional media production 
project and which are less so. 



HCI component 


Number of 
Instances 


Images 


12 


Text 


12 


Overt Buttons ('standard' 
buttons) 


10 


Age Appropriate Font Size 


10 


Linear Navigation 


9 


Feedback (correct and wrong) 
Per Question 


9 


Consistent "Look and Feel" 


9 


Images that Act as Buttons 


8 


Output Fields 


8 


Local Variables 


8 


Well-Crafted Text Content 
(well-written text) 


8 


Allowing a user to continue (if 
the user answers correctly, they 
may go on) 


7 


Input Fields 


6 


"Next" and "Back" Buttons 


6 
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Multiple Choice Questions 
Grammar, Spelling, Puncuation 
(high quality) 

Animated Text 



6 

6 

5 



Triggering a dialogue box ("ycm| 
didn't fill in your name") 

Feedback for Completed Test 
(Test Score) 

Short -answer 
Video Sequence 
Sound effects 
Non-Linear Navigation 
Global Variables 
Animated Buttons (rollovers) 

As Part of Video Sequence 

Sequenced with Animated Text 

or Graphics 

Basic Math Functions 

Used for stories 

Drag/Drop 

Highlight keywords 

Background Music 

Writing function 

Review function 

Animated Images 

Screen Changes (transitions) 

Menus 

Narration 



5 

5 

4 

4 

4 

4 

3 

3 

3 

3 

3 

2 

2 

1 

1 

1 

0 

0 

0 

0 



Table 2: Numbers of instances of each HCI component in the 12 projects examined 



Limitations 

Although we are intrigued by the findings of this preliminary study, it must be constantly borne in mind 
that this is indeed only a preliminary study. While the projects examined represented both a wide range of student 
achievement and a range of varying content areas (from kindergarten activities to corporate training), we only 
examined 12 projects in all (a very small number considering the universe of projects created for such courses 
worldwide). 

Also, for the sake of convenience, we only examined projects for which the authors served as course 
instructor (7 of the projects were created for one author’s courses; 5 of the projects were created for the other 
author’s courses). To reach any conclusions that might be applied to all Instructional Design Technology programs, 
we would have to expand the examination to projects created for a great many other instructors in similar programs. 

Discussion 

The modes and levels of human-computer interaction presented in this paper are intended only to reflect the 
authors’ initial thinking on the subject. While we believe it to be a reasonable starting point for a discussion, it is 
only that at this time. Like Bloom 's Taxonomy of Educational Objectives ( 1956), this organization of HCI elements 
is meant to spur further discussion. Unlike Bloom’s Taxonomy, this is not yet a popular view of the HCI elements. It 
requires the feedback from many more experts in the field before it can be modified to a point that allows members 
of the community to generally agree upon its usefulness. 
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To continue to discuss what the next steps might be in our study, we feel it is important to examine a great 
many more projects. We are considering the merits of applying a more specific statistical analysis to the results of 
that examination (e.g. a factor analysis might prove enlightening). 

Perhaps the judgment of individual faculty members sophisticated in current advanced technologies may be 
combined with information about those aspects of HCI that are important to their colleagues in a variety of well- 
respected programs. There may be some merit to the creation of a set of standards for human-computer interaction 
skills as they are applied to instructional media production. Even if the creation of standards is not the answer, 
expanded analysis of which HCI considerations merit the greatest attention in an instructional media production 
course seems prudent at this time. 



Conclusions 

The examination of the 12 student projects using the list of HCI categories, sub-categories and specific 
instances suggests that all computer-based instructional media projects include some combination of images and 
text. The second most important HCI considerations for such projects include the use of overt buttons, control of the 
font size, a linear navigation component, feedback for questions posed and a consistent look-and-feel. These HCI 
considerations might well form the basis for what is considered beginner- level control of computer-based 
instructional media. 

Also important, but to a lesser degree, are HCI considerations that include: images that act as buttons, 
output fields, the control of local variables, and well- written text. These, combined with the beginner-level HIC 
considerations, might form the basis for intermediate control of computer-based instructional media. 

Those HCI considerations that occur in half or less of the sample population might be considered “extra” or 
“embellishment” or “optional” considerations that are applied if the context of the instruction demands them. They 
might not be considered essential elements that all novice instructional media producers would have to have control 
over, and their presence in a project (along with the HCI considerations that comprise intermediate control) might 
indicate advanced control of computer-based instructional media. 

Regardless of whether the label of beginner, intermediate or advanced is applied to specific instances of 
HCI, the very fact that certain HCI considerations are observed in many or all of the projects examined suggests that 
learning to control these HCI considerations should be a part of any course of study in Instructional Design 
Technology. 
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Abstract 

As part of a Preparing Tomorrow’s Teachers to Use Technology (PT3) grant, the authors are working on 
the design, development, and evaluation of an online support environment for novice teachers and teacher support 
specialists/mentors that will continue to expand technology integration support throughout the critical student 
teaching and teacher induction phases. This paper describes the project context; a review of selected electronic 
support systems for teachers; an early prototype; and a proposed design approach. 

Context 

The Crossroads: Preparing Tomorrow’s Teachers to Use Technology at the Intersection of Content, 
Pedagogy, and Technology project at Georgia State University is a grant project sponsored by the U. S. Department 
of Education. Resulting from an analysis of technology integration needs in Georgia and Atlanta Metro schools, the 
goal of the project is to “prepare technologically proficient teachers for Metropolitan Atlanta’s many diverse 
classrooms by addressing the intersection of content, pedagogy, and technology with training, support systems, and 
environmental change’’ (The Crossroads, 2003, p.3). One of the grant objectives is, “to continue to expand 
technology integration support for our students and graduates throughout the critical student teaching and induction 
phases through the design, development, and evaluation of an online support environment for novice teachers, and 
teacher support specialists/mentors.” (The Crossroads, 2003, p.3). That is, the authors were charged with designing 
an online environment that would expand technology integration support for novice teachers throughout their first 
years of teaching, one that would be situated within their actual teaching experience to help nurture their lifelong 
learning and reflective practice. The system would be able to help connect student teachers, their mentors, teaching 
resources, and evidence of best practice in one virtual space. This support system would also remain available to 
student teachers throughout their induction phases in order to help nurture lifelong learning and reflective piactice. 
The original model proposed for this environment was based on the concept of electronic performance support 
systems (EPSS), which are more commonly discussed in a corporate context. 

An EPSS is an electronic system that can provide integrated, on-demand, just-in-time, individualized on- 
line access to information, expert consultations, learning experiences, tools, assessment and monitoring systems to 
enable a high level of job performance with a minimum support and intervention by others (Brown, 1996; Gery, 
1991, 1995a; Raybould, 1995; Winslow & Bramer, 1994). Gloria Gery has been credited with initiating EPSS 
movement in 1991 with her book. Electronic Performance Support Systems: How and Why to Remake the 
Workplace Through the Strategic Application of Technology. Gery and other EPSS pioneers have suggested that 
building and implementing an EPSS can help organizations in corporate and educational settings shift their 
paradigm from training to performance, from viewing performers as people to be trained to viewing them as people 
who need on-the-job performance support (Brown, 1996; Rosenberg, 1995). Most EPSSs consists of four 
components : 1) advisory component, 2) information component, 3) training component, and 4) user interface 
component. However, a simple prescription or a standard set of features for building an EPSS does not seem to 
exist. As Gery (1995b) put it, “few (designers) are guided by a set of integrated and fully articulated design 
principles. Many innovations are the result of individual or team creativity and iterative design employing rapid 
prototyping coupled with ongoing usability and performance testing.’’ The presenters were drawn to the EPSS model 
because it seemed an appropriate mechanism through which a wide range of support could be provided to novice 
teachers situated within the context of their actual practice. 

Method 

The authors began by reviewing the needs analysis and resulting goal and objectives of the Crossroads 
grant mentioned in this paper. Formal and informal discussions then took place between instructional design experts, 
teacher education experts, performance technology experts, and teacher education faculty stakeholders. An extensive 
review of existing systems was also conducted using the following criteria: a) The system was created for 
educational purposes; b) the system was intended for PreK-12 or K-16 pre -service teacher education and in-service 
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teacher professional development; c) the system addresses technology integration in the curriculum; d) some EPSS 
components as defined above are present. The review includes both commercial products and systems developed for 
public access. Seven systems were selected and are briefly described below. 

Some Selected Electronic Teacher Support Systems 

Knowledge Innovation for Technology in Education (KITE) is a PT3 project at the University of Missouri. 
Its development involves the collaboration of a consortium of eight teacher education programs. Its overarching 
mission is to “build a K-16 community of practice through a knowledge repository that enables learning through 
sharing, communal understanding through storytelling, continuous exchange and creation of new knowledge, and 
collective problem solving among K-12 schools and teacher education programs” (KITE, 2004). 

KITE consists of four main components: Introduction, Technology Integration Cases, Technology 
Integration Learning Environment, and Teacher FAQ. The Introduction section presents information regarding; a) 
KITE’S general project information; b) university partners and key players involved in the project; and c) brief 
introductions and web links for other technology integration resources. As an integral component of the project, 
KITE’S introduction presents end-users with project background information. The Technology Integration Cases are 
the primary content of KITE. Each text -based case has two sections: case summary and whole story. Each case 
depicts an interviewee’s experience with his/her technology integration practice. The Technology Integration 
Learning Environment (TILE) provides information and guides related to topics such as national education 
technology standards (NETS), media selection, lesson planning, assessment of technology integration experiences, 
and creation of a teaching unit. It serves as a supporting pedagogical framework for technology-integrated and 
project-based instructional activities espoused by KITE cases. Teacher FAQ provides: a) Descriptions of 
instructional activities conducted in several teacher education programs; b) introduction of some ideas on how to use 
KITE cases in teaching technology integration; and c) answers given to the KITE related questions. KITE’S 
structure, especially that of TILE helped inform the design of the authors’ system prototype. 

Integrating New Technologies into the Methods of Education (INTIME) is a PT3 project at University of 
Northern Iowa, created by a consortium of five teacher education programs. Guided by the Technology as Facilitator 
of Quality Education conceptual model (TFQE) (Callahan & Switzer, 2001), INTIME aims to help teacher 
education programs improve pre -service teachers’ knowledge and skills to effectively integrate technology in the 
PreK- 12 classroom. 

INTIME has five main components: Streaming Video Cases Build a Case Study, The TFQE Model, 
Multicultural Education, and Help. Streaming Video Cases are the primary content in INTIME. The cases are all 
technology integration cases in real PreK- 12 classroom settings. Each case includes a detailed lesson plan, nine 
video clips each with narrations and annotations, and pre -viewing and post-viewing probing questions. Build a Case 
Study is an online learning tool where a student teacher can use INTIME’s video clips to make his/her own case 
study to facilitate technology integration learning. The tool provides a step-by-step guidance on how to create a 
printable case study. This is an innovative and easy-to-follow tool whose concept the authois may borrow. The 
TFQE Model includes seven major dimensions. It serves not only as a conceptual framework for the creation, 
segmentation, and presentation of online streaming video cases, but also as search criteria of the video cases. 
Multicultural Education provides information and helpful guidance in various aspects such as school-wide 
multicultural benchmarks and characteristics considerations, studying of ethnic and cultural groups, curriculum 
consideration, and more. The Help component provides technical instructions for using INTIME. Unlike the text- 
based case presentation in KITE, the video cases in INTIME proved to be a more desirable model for the authors’ 
system. 

EduCatalyst is a standards -based online performance support system, which allows “schools, districts, and 
states to determine each teacher’s strengths and challenges, design individually professional development plans, 
develop and implement innovative, measurable learning resources, and benchmark student performance and 
evaluate the effectiveness - both academic and financial of their professional development efforts” (EduCatalyst, 
2004). 

EduCatalyst consists of six major components: Portfolio, Professional Development Alignment Catalyst, 
Planning Catalyst, Instructional Strategy Catalyst, Assessment Catalyst, and Support. Portfolio provides a digital 
locker where teachers can store their transcripts and works, and store and track their licenses and re-certification 
progress. The Professional Development Alignment Catalyst helps teachers align their professional development 
activities with the priorities within their state, district, or school, and view the lists of professional development 
programs, courses, and classes made available to them. Planning Catalyst allows teachers to create, store, and 
complete professional development plans, and to review the recommended plans. Instructional Strategy Catalyst 
provides teachers with access to various effective education resources and products. Assessment Catalysts allows 
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teachers to conduct high-quality assessments of student performance either with the help of rubrics database 
provided or by creating a rubric from the scratch. Support provides the educators with access to: a) Printable version 
of the help system and video tutorials; and b) on-line help which is composed of three search mechanisms - tree- 
structured contents, index, and a keyword search. 

EduCatalyst boasts a few powerful features that helped inform the authors’ design. First, EduCatalyst 
emphasizes the construction of teachers’ professional development activities by linking them with various levels of 
educational standards. Second, EduCatalyst espouses cultivation of the educators’ experiential knowledge and skills 
for professional development, as evidenced by their hands-on practice on creating action plans. Third, EduCatalyst 
maintains an online personal account that not only serves as an e-portfolio, but also governs the conduction of their 
different catalyst activities. 

Knowledge Loom is a knowledge sharing community created and maintained by Brown University. It is 
organized around best practices in teaching and learning, and has evolved into “a comprehensive electronic 
environment that moves from information delivery to information creation, from data to people, from a learning 
library to a learning community” (Knowledge Loom Team, 1999). 

Knowledge Loom includes two major components: Collection and Spotlight Library. Collection contains 
descriptions of best practices that can be sorted by theme, state, grade, setting, school type, and school 
design/organization. Spotlight Library provides information about specific topics such as teaching with technology, 
middle school math, professional development, and etc. Within a specific Spotlight such as adolescent literacy, 
contents can include: a) Key components of a successful adolescent literacy initiative and the best practices related 
to each component; b) successful strategies; c) research that identifies and supports the best practices; d) stories 
about the best practices in real schools and districts; e) subscription of ones own stories, open-ended questions with 
experts in the area, and threads of wisdom; f) presentation of the threaded online panel discussions; g) list of 
supporting organizations and resources; and h) spotlight contents to download. 

As an ongoing virtual learning community, Knowledge Loom not only maintains an excellent mechanism for 
presenting and soliciting best practices, but also facilitates multiple layers of knowledge creation and sharing 
mechanism - both of which helped inform the authors’ design. 

Teachscape aims to build a virtual professional development community for teachers and educators. 
Teachscape advocates research-based, classroom-tested approaches to effective teaching. It adopts video teaching 
cases to support new teachers’ existing induction programs to enhance their capabilities to embrace the 21st century 
challenges: “new high standards, accountability for school, and new technology” (Teachscape, 2004). It also offers 
experienced teachers with a range of courses helpful for their certification or re -certification requirements, as well as 
a mechanism for their coaching and mentoring of new teachers. 

Teachscape consist of four main components: My Desk, Resource Library, Public Discussions, and Journal. 
My Desk lists general account information and personal homepage where assigned materials are provided. Resource 
Library is the primary content in Teachscape, which features content knowledge in the areas of literacy, 
mathematics, science and class management skills. Each course incorporates video cases, expert commentaries, 
discussions, written materials, and research related to the course topic. Video cases integrate rationale and 
background of the case, national and local standards in the content area, explore different specific topics, and 
provide teacher reflection and specialist commentary. Public Discussions allow the teachers to participate among 
their learning groups and in public arena. Journal functions as an e-locker where the teachers can write, store, and 
keep track of their reflections on teaching and learning. These features all helped inform the authors’ design. 

Elementary and Secondary Teacher Education Project (eSTEP) at the University of Wisconsin is an 
innovative, experimental web-based approach to teaching pre -service teachers on how to acquire current scientific 
knowledge about human learning and development in educational settings - learning sciences (eSTEP Team, 2002a; 
Sharon Derry and the STEP Team, 2002). 

eSTEP integrates three main instructional components in its online tool: Knowledge Web (KWeb) Cases, 
KWeb Theories, and Problem-based Learning Online. KWeb is an online multimedia environment that interlinks 
video cases and problems with learning science theories and research. KWeb Cases are a collection of classroom 
problems and multimedia cases depicting stories of teaching and learning in authentic classroom settings. KWeb 
Theories list learning sciences theories and research from cognitive psychology relevant to teachers, to facilitate 
teachers’ exploration of various theoretical perspectives on teaching and learning. Problem-based Learning Online 
facilitates individual learning, group collaborative work, and use of the eSTEP Knowledge Web. Through pbl 
activities, students encounter problems structured around video cases of actual instruction, collaboratively analyze 
the instruction from a learning sciences perspective, and redesign or adapt the instruction based on their analyses 
(eSTEP Team, 2002b). 
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Created by a consortium of nine high schools, the University of West Florida, and the Navy Training 
Network, Support for Teacher Enhancing Performance in Schools (STEPS) is a World Wide Web and CD-ROM 
EPSS designed to provide “just-in-time” support to help pre- and in-service PreK-12 teachers develop instructional 
lessons, units, and curricula aligned to Florida’s Sunshine State Standards. STEPS embodies Florida school reform 
and accountability initiatives in four areas: integrated curriculum, integrating technology, alternative assessment, 
and diverse learning environments (Northrup & Pilcher, 1998). 

STEPS consists of six main components (Park, Baek, & An, 2001): Lesson Architect, Tutorial Library, 

Best Practices, Sample Unit, Web Links, and Coach. Lesson Architect guides teachers through the processes of 
instructional design and curriculum planning. Tutorial Library provides about forty instructional tutorials, following 
the aforementioned four areas of school reform and accountability initiatives. Best Practices provide teachers with 
multiple search mechanisms for successful classroom activities developed and tested by teachers in their real 
classroom settings. Sample Units provide sample curriculum units created by teachers of the same grade level. Web 
Links list more than 400 web sites relevant to Math, Science, Social Studies, and Language Art identified in Florida’ 
Sunshine State Standards. Coach facilitates three levels of scaffolding: the “big picture” level, the “what do I do” 
level, and the “how do I do” level. The authors’ design was informed by the Lesson Architect. Best Practices, and 
Coach features of STEPS. 



Initial Prototype 

As a result of ongoing analysis, review, and related discussions with students and colleagues, the authors 
designed an initial prototype for a Teachers’ Electronic Performance Support System - TEPSS. See Figure 1. The 
Information component of TEPSS consists of links to databases containing lesson plans, instructional resources, 
assessment tools, and more. The Training component of TEPSS will provide users with access to learning modules, 
and job-aids related to their practice. The Tools component of TEPSS will provide users with activity design and 
student-teaching video reflection “wizards” (not unlike software help wizards) that would guide the novice teacher 
through planning an activity and analyzing video footage of themselves implementing activities using a series of 
textual prompts. The Guidance component of TEPSS would consist of a communication forum, (perhaps using 
blogging technology), and a series of guiding questions and answers concerning how to use the system. The current 
conceptual model (prototype) may likely change based on our design approach and resulting stakeholder feedback. 

Continued Design and Development 

Rapid prototyping is defined as “the creation of a working model of a software module to demonstrate the 
feasibility of the function. The prototype is later refined for inclusion in a final product” (Webster dictionary). Rapid 
prototyping dates back to the mid-1980s, and has been widely adopted in manufacture engineering and software 
development. Recent years have witnessed the increasing influence of rapid prototyping design methodology in 
instructional design, especially for computer-based instruction (M. K. Jones, Li, & Merrill, 1992; T. S. Jones & 
Richey, 2000; Lohr et ah, 2003; Rathbun & Goodrum, 1994; Tripp & Bichelmeyer, 1990). Tripp and Bichelmeyer 
(1990) provide a useful description of rapid prototyping in the context of instructional design; “after a succinct 
statement of needs and objectives, research and development are conducted as parallel processes that create 
prototypes, which are then tested and which may or may not evolve into a final product” (p. 35). Prototypes for the 
current project will be both paper-based and computer-based including “any required databases, the major program 
modules, screen displays, and inputs and outputs for interfacing systems” (Harmon, Reece, Shoffner, Calandra, & 
Dias, 2003; Tripp & Bichelmeyer, 1990). Presenting these models to experts and system stakeholders will serve 
four main purposes: a) to obtain buy-in from faculty, student and administrative stakeholders; b) to solicit feedback 
on tasks, content, and features of TEPSS; c) to gain development feasibility feedback from programmers; and d) to 
obtain implementation feasibility feedback from system administrators. The design, feedback and development 
loops should generate larger, more complex, and more robust prototypes with each iteration. The design of each 
prototype will be directly affected by feedback regarding the one previous until a satisfactory final product has been 
designed/developed. See Figure 2. 
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Three Levels of Motivation in Instruction: Building Interpersonal Relations 

with Learners 

Katy Xinquan Cao 
Indiana University Bloomington 

As teachers and instructors, what role do you think you should play in the students’ lives? Take a minute to 
think about it, did you have a professor that you feel what you learned from him and his class shaped your life? On 
the other hand, was there a professor that you never went back to him again? The teachers’ goal should not simply 
be to teach the items on the curriculum, but also to be an example as a person and a respectable scholar for the 
students. What kind of scholar you are and what you offer in your instruction are important motivation for the 
students, and will impact the students’ lives tremendously. The ties between teachers and students are loaded with 
emotions and responsibilities. 

Based on the emotions and responsibilities, this paper proposes a model that identifies three levels of 
motivation (3LOM) in instruction. It states that motivation can be addressed at three different levels: inclusion, 
entertainment and edification. It looks at motivation from an angle of social interaction. The focus of the model is to 
describe teacher’s role as an active party in the process of teacher-student interaction. The assumption is that ideal 
instructional interaction in class, as with any other types of social interaction, should attend to, and indeed give 
priority to the students' certain needs and desires. Otherwise, it will turn into a bad experience that the participant 
does not want to repeat. These needs and desires are in a hierarchical order and labeled “inclusion, entertainment, 
and edification.” 

The following are the values underlying this model: 

• The purpose of instruction should serve the positive needs of the society and promote the development of 
the society. 

• The instructor should first of all have sufficient expertise and good qualities or standards that are acclaimed 
by the majority of the society. 

• The process of instruction and learning is one type of social interaction that should be carried out 
accordingly. 

• The instructor should seek to understand the needs of each student. 

• The instructor’s first priority is to teach the things as listed in the curriculum. 

• Secondly, if s/he can, the instructor should explore the learner’s potential and provide guidance for the 
learner to achieve his potential in the future. 

• Learners have their own free agency. The instructor is not to force changes on them but cater to their 
individual potential and ambition. 

An important stimulus of the model is Abraham Maslow’s hierarchy of needs. The way that the three levels 
are organized as a hierarchy is exactly following Maslow’s hierarchy of needs. The highest level, edification, is the 
educational equivalent of Maslow’s ideas of self-actualization and self-transcendence. Other reference to literature 
in instructional design principles, motivation and good practice in classroom have also contributed to the 
development of the three levels in this model of motivation. 

Maslow’s Hierarchy of Needs 

The 3LOM model developed its framework from Maslow’s (1943)hierarchy of needs, which stated that 
“human beings are motivated by unsatisfied needs.” Those needs are in a hierarchal order with “certain lower needs 
need to be satisfied before higher needs can be satisfied.” 

In the hierarchy, Maslow included some general types of needs (physiological, safety, love, esteem, and 
self-actualization). He argued that these “needs must be satisfied before a person can act unselfishly.” He called 
these needs "deficiency needs. As long as these cravings are satisfied, human beings will move towards growth, 
toward self-actualization. Satisfying needs is healthy; blocking gratification makes us sick or evil.” (Maslow, 1943) 
“Needs are prepotent. A prepotent need is one that has the greatest influence over our actions. Everyone has a 
prepotent need, but that need will vary among individuals.” (Maslow, 1943)An actress may have a need to feel that 
her change of image is liked by the audience. A prisoner will need to satisfy his cravings for freedom and will not 
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worry about his look. In a school setting, a professor may need students’ respect and attendance in class. A student 
may feel that he needs to keep up with the rest of the class. Maslow’s theory provided an effective tool that helps us 
understand ourselves as human beings. 

Based on Maslow’s theory, we should view instructional interaction as a social process in which the 
learners are motivated by certain needs to be active. 

Basic Principles of Instructional Design 

The three basic principles in instructional design are that: instruction should be appealing, effective and 
efficient. (Smith & Ragan, 1999) All the three can be considered as constructs of motivation. Lacking of any one of 
the three is likely to result in discouraging students’ interest in the instruction. The 3LOM model aims to help 
achieve these goals in instruction. 

Related Literature on Classroom Motivation 

Literature on methods of motivation and classroom practices were referenced to make the 3LOM model 
more applicable. Two of the writings are especially enlightening. 

C. J. Bonk and V. P. Denn's (2001)article We ’ll Leave the Light On For You: Keeping Learner Motivated 
in Online Courses provided some very detailed tips that are implementation-oriented. Bonk and Denn summarized 
ten points that are motivating in online classroom: tone/climate, feedback, engagement, meaningfulness, choice, 
variety, curiosity, tension, peer interaction, and goal driven. 

The 3LOM model is in agreement with the above article. All the ten points map directly onto one or more 
of the three dimensions of 3LOM model. For example, Tone/climate and peer interaction are absolutely important in 
inclusion. Engagement, variety and curiosity are all good methods that will build up the dimension of entertainment. 
Feedback and goal driven are methods for edification. 

Chickering and Gamson (1987) in their Seven Principles for Good Practice in Undergraduate Education 
listed the following ideas: 1. Encourages contacts between students and faculty. 2. Develops reciprocity and 
cooperation among students. 3. Uses active learning techniques. 4. Gives prompt feedback. 5. Emphasizes time on 
task. 6. Communicates high expectations. 7. Respects diverse talents and ways of learning. These seven principles 
also can find their counterpart in the 3LOM model. The first principle, “encourage contacts between students and 
faculty refers to inclusion’’ and the second one, “develops reciprocity and cooperation among students” refers to 
inclusion. The third one, “using active learning techniques” is more about entertainment. The fourth, sixth and 
seventh principle all fall into the dimension of edification. 

Although the 3LOM model has many things in common with the other two articles, there are some 
principle differences too. First of all, as stated at the beginning, the 3LOM model looks at the issue of motivation 
from a social psychological point of view. It emphasizes that motivation can be gained at three different dimensions 
instead of many isolated points. 

Secondly, 3LOM model implies that the teacher is an active party that is aware of the students’ needs in the 
teacher-student interaction process. The teachers take the initiative to satisfy those needs and make the teacher- 
student interaction alive. 

Level of Inclusion 

Inclusion is a prerequisite for social interactions. We need to be included in a group to carry out 
communications or interactions. One implication of inclusion is unconditional acceptance. The student should feel 
that he is welcome simply because he is a student in the class. The teacher should be careful not to be judgmental 
with the students when they first meet. Unconditional acceptance is the door opener for the new comer. To include 
people in, we give them signals by responding to their presence and paying them attention, respect or care. This is to 
satisfy the learner’s social or emotional needs. As Edgar Dale wrote in his book Audio-Visual Methods in Teaching 
(1946), “Good teaching involves the feelings as well as the intellect.” In the class, if a student felt ignored or 
rejected in his attempt to start a conversation, or came across some unfriendly comments, he would not feel the 
atmosphere was comfortable for him to be there. An instructor should make sure to include all students in the 
learning process. 

Virtual inclusion can be achieved through two types of relations built in the instruction - learning process: 
first, a positive personal relationship between the instructor and individual learner; second, spirit de corpora in the 
classroom. The earlier these relations are formed, the better they will help the instruction and learning process. 

Edgar Dale wrote some similar ideas about this years ago. He used the word “mood of mutuality.” As he said, 
“Learning blossoms in a mood of mutuality. Such a mood must permeate the classroom, the shop, the home, or 
wherever else teaching takes place, if it is to be good teaching.” (1946) 
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Building a positive personal relationship between the teacher and the learner A positive personal relationship 
between the instructor and the learner should be a two-way relationship that includes two facets: the instructor 
genuinely cares about the learner and the learner trusts and looks to the instructor for guidance if he feels he needs 
any. For the instructor to form a personal positive relationship with the individual learner, the instructor should 
prepare to build rapport with the individual learner from the very beginning. The instructor should target to gain the 
learner’s trust as soon as the learner appears in his world, in class as well as after class. It’s important that this 
friendly relation be built in the first few contacts because first impression is a very powerful factor that shapes our 
perceptions of other people. However, this doesn’t mean that the teacher totally lost the chance if s/he did not 
become friends with the students in the first place. But an early friendship is more preferable. 

The following methods can be used: 

1. Know the students personally 

This includes remembering students’ names and addressing their names clearly, chatting with the students 
to find out some relevant background information, such as their interests and career goals. 

2. Show appreciation in students’ abilities 

Teachers should know clearly how the students perform in the class. It is important for the teachers to spot 
one or two things that a student is good at and show sincere and generous appreciation. Teachers should also be 
understanding. If a student is not doing well, the teacher should find out the reasons and help with solutions. 

3. Be available when students need help. 

Besides the teaching-learning interactions in class, teachers should also maintain good communications 
with the students after class, such as keeping regular office hours or replying emails carefully and timely if students 
contact him. The point here is the students should get the idea that the teacher is approachable and ready to help 
when needed. 

Maintaining spirit de corpora in the classroom For a student, unpleasant feelings with one or two peer students in 
the class are likely to ruin the whole learning experience. Teachers should be careful not to encourage any cleavage 
between students consciously or unconsciously. Often times, top students received much of the teacher’s favor while 
problematic students obtained much of the teachers attention either in a positive or negative way. It is easy for 
students to feel that they have different status in the clas s. Negative feelings such as jealousy or contempt toward 
each other among students are not likely to help with students’ growth and should be prevented. 

1. Teachers should treat every student equally. 

2. Peer students should be encouraged to respect and help each other before they compete. Students should 

also be encouraged to remember each other’s names. 

3. Learning groups can be formed and shuffled regularly. 

4. Fair group work norms should be formed in the class. 

Level of Entertainment The idea of entertainment means making the learning process fun and relaxing instead of 
boring and frustrating. The point is to remove the fatigue or stress resulted from the intense intellectual process in 
the learners’ brain and help the learners to concentrate on the learning easily and as long as possible. 

There are three aspects to consider when designing entertaining instruction: choosing entertaining learning 
materials if possible, using entertaining delivery methods and the instructor personally developing entertaining 
teaching style. These three methods are complementary to each other. The instructors should use them in proper 
situations. 

1. Choosing entertaining learning materials 

If the instructor has the authority to decide what materials to use, he can choose learning materials that are more 
appealing to the learners, such as using books written in a language that is more colloquial rather than deep and 
complicated professional prose. 

2. Using entertaining delivery methods. 

Most of the time, instructors do not have mu ch freedom to choose easy learning materials. If this is the 
case, the instructors can work on making the delivery methods entertaining. 

Most popular examples of entertaining delivery methods are the use of multi-media such as video, audio, 
graphics, games and many other creative ways to make the instruction a fun thing. For example, many professors 
bring refreshment to their classroom or arrange students to bring food once in a while. Others would start the class 
by playing a little piece of lively music. These different strategies all serve one purpose, which is building a relaxing 
or refreshing atmosphere in the classroom. 

3. Teachers developing entertaining personal teaching style 
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A teacher’s teaching style is much related to his or her personality. Some people have the talent to be 
humorous, witty or funny when they talk, such as some talk show hosts or hostesses and comedians. Others might be 
born with an easy going and happy temperament that lighten up things. These are all places that a teacher can 
explore to make his instruction entertaining. Telling jokes is a method that can help a teacher to be entertaining. 
However, the jokes should be relevant to the instruction, clean in both the language and ideas. In another word, it 
should be good humor. The instructor should have some skills in telling jokes too. Otherwise, the jokes will not 
work. If used wisely, the jokes can greatly help the learners understand and remember the learning material. 

It is hard to sum up all the methods of entertaining instruction in class. Different people have different 
strategies. Depending on the specific circumstances, the instructor should adopt the appropriate method to cheer up 
the learners. 

Level of Edification 

Literally, edification means intellectual, moral, or spiritual improvement. What this paper is trying to say is 
that in the instructional process, a learner experiences revolutionary upgrade in his understanding in one or all of the 
above domains, which results in changes in his way of thinking and/or behaviors in a better way. 

Intellectual edification deals with our understanding of the objective world. In the process, we move closer 
and closer towards the truth. Examples are that we feel good when we are exposed to the rules about how things 
exist and evolve in the world. For example, people are no more afraid of eclipse when they understand what causes 
it. 

Edification in the spiritual and moral domains deals more with our life views. It helps to answer questions 
such as “What is the meaning of life?” and “What is a good way to lead this life?” 

The strategy recommended for edification is self-actualization, and self-transcendence for spiritual and 
moral development. Self-actualization means to “become more and more what one is, to become everything that one 
is capable of becoming.” (Maslow, 1943) Self-actualization was used by Maslow in his hierarchy of needs. 
Edification in instruction should be aligned with each student’ s self-actualization to be most motivating. However, 
helping the learners to achieve self-actualization is not exactly what edification means. The instructor should first 
make sure that the learners’ self-actualization will benefit but not harm the world. Otherwise it is not edification. 

The reason is simple. We do not want to help somebody to become another Adolph Hitler. Also, the use of the ideas 
of self-actualization and self-transcendence in this paper only refers to people’s attitudes about living a meaningful 
life. Practices such as using drugs to help human beings’ perception go beyond normal to realize self-transcendence 
are not the concern of this paper. 

Edification in the intellectual domain Intellectual edification means the increase of knowledge about the objective 
world, which includes us as human beings, nature, and the relations and rules of the objects in the universe. At a 
basic level, the learners should be exposed as much as possible to general knowledge and skills, which is to allow 
them to function well and handle the problems in daily life. At a second level, they should acquire some awareness 
of their own strengths and weaknesses. Based on this, they will be able to focus on their strengths and fully develop 
their potential. The result of the basic level is that students should have the confidence to say that “I am as good as 
anybody else.” Besides, some students should be able to say “I am really an expert in this area.” As a matter of fact, 
due to one reason or another - either biological or social factors - it is not reasonable to expect every student to 
reach this level. But the teacher should be alert to detect those students who are exceptional in one way or another 
and help bring out the best of those students, 
a. Acquiring general knowledge and skills. 

General knowledge and skills have been receiving much attention in education. It is not new idea. One 
example is K-12 education. What the students are expected to learn through K-12 years are mostly focused on 
general knowledge and skills. In almost every country and educational system exists a curriculum that defines the 
goals and objectives of the K-12 education. The curriculum specifies what kind of skills and knowledge every 
student is expected to obtain in his or her school years. Those skills and knowledge focus on different subjects: 
Language, Mathematics, Arts, History, Physics, Chemistry, Geography, Science and so on. To better understand 
all the different general knowledge and skills, let’s borrow Gardner’s (1993a)theory of multiple intelligences, 
which describes, 

. . .all human beings represent the culmination of an evolutionary process that has yielded at least eight 
relatively discrete information-processing mechanisms. All of us possess linguistic intelligence; logical- 
mathematical intelligence, musical intelligence; spatial intelligence; bodily- kinesthetic intelligence; 
naturalist intelligence; interpersonal intelligence; and intrapersonal intelligence. 
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Based on multiple intelligences, some general knowledge and skills students should have are: the capacity 
to observe and gather useful information, the capacity to effectively communicate with other people orally and in 
written form, the capacity to maintain good relationship with people around, the capacity to understand the 
emotional world of oneself and other people, the capacity to make sound judgments on things happened and react 
properly, the capacity to understand nature and the universe, and the capacity to appreciate different types of arts. 

b. Help students becoming expert in an area (or areas) of interest. 

Teachers should pay enough attention to students so to detect students’ strengths or potential. After 
communicating with the students about their interests or ambition, teachers should provide guidance in 
determining an area of interest and further development. In other words, teachers should help students to be aware 
of their strengths and potential. Students should be encouraged to become expert in one or more areas of interest 
or whatever their potential is. 

Because of limited opportunities of contact or large number of students to work with, it might not be easy 
for a teacher to know students well enough to determine their potential area. Some other ways can be used to help 
the teachers. First of all, teachers should get involved with students as much as possible; second, the students 
should be encouraged to exhibit their strengths whenever there is a chance; third, teachers should work closely 
with students’ parents or families to learn more about the students. 

2. Edification in moral and spiritual domains 

There are two levels in moral and spiritual edification: self-actualization and self-transcendence, 
a. Self-actualization 

As stated earlier, Maslow (1943) defines that “the need for self-actualization is the desire to become more and 
more what one is, to become everything that one is capable of becoming.” One key question here is “How can one 
know what he is capable of becoming?” There are two possible ways an individual finds out about his potential: 
personal awareness and external influence. Personal awareness is mostly developed as individuals accumulated 
life experiences. In the process, they gradually come to realize his or her potential and become determined to fully 
develop it. External influence is from individual’s community or society, such as a certain way of living valued or 
some social standards in the culture. As the individual grow up in the culture, he or she is expected to achieve it 
as an ideal. 

Those who came to know their potential through personal awareness are most likely “People who have 
everything can maximize their potential. They can seek knowledge, peace, esthetic experiences, self-fulfillment, 
oneness with God, etc. It is usually middle -class to upper-class students who take up environmental causes, join 
the Peace Corps, go off to a monastery, etc.” (Maslow, 1943) However, the appreciation and support from their 
family, friends or mentors might not be indispensable but important. 

A typical example of living up to the standards of the society to carry out self-actualization is found in 
Confucianism, a traditional Chinese philosophy, which is also understood as scholar’s philosophy in China. 

In the past, all scholars (man only) in China were taught to go through a same life ladder. First, a man should 
do self-cultivation, which means that he should seek education and cultivate good qualities. Second, a man should 
raise a good family and maintain a harmonious household. This implies that men should be good husbands and 
fathers. Third, a man should use his wisdom and skills to serve and govern his country, like obtaining a prominent 
position in the emperor’s court. Fourth, a man should aim to make world peace. 

Nowadays, people have much freedom to choose the things they like. Edification can happen in many 
different ways. Cultural background such as life style and values is an important factor that influences self- 
actualization, 
b. Self -transcendence 

Self -transcendence means to “connect to something beyond the ego or to help others find self-fulfillment 
and realize their potential.” (Maslow, 1943) 

Examples of this are mostly found in different religions. In Christianity, Jesus Christ is regarded as the 
savior of the world. He sacrificed his own life for the world. A recent example would be Mother Teresa, who lived 
and worked to promote the happiness of a people in a strange land. In some western churches, people are 
encouraged to become like God and Jesus Christ. In Buddhism, it is believed that everybody can become a Buddha, 
who is regarded as saint and almighty because he is completely unconnected with, therefore unrestrained by the 
secular world. 

The above are some areas that an instructor might look to edify his learners morally or spiritually. A key 
idea with the results of edification should be changes in a learner’s way of thinking, behavior or both. Otherwise, 
edification does not happen on the learners’ side. 
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3. Methods to make edifying instruction 

a. Choose meaningful, relevant and applicable learning content. 

Learning materials should be related to learners’ careers, goals or something that is part of the students themselves 
or their life. 

b. Encourage self-reflection 

Encourage students to check their own progress. The students should learn to learn from their own experiences 

and be able to adjust their goals. 

c. Provide intervention when necessary 

Teachers should be aware of students’ development and be prepared to give further guidance when needed. 

Inclusion, entertainment, and edification are the three things a teacher can look at to make his instruction 
motivating. They are hierarchical in the way that they motivate the learners at different levels based on the needs 
and desires as expected in the psychological process of human interaction. The levels of inclusion and 
entertainment are important factors to foster learning, however, these two are not the final goal of instruction. They 
help to achieve the third level of edification. Without reaching the level of edification, inclusion and entertainment 
do not have much significance in an instruction-learning interaction. Also, there are no absolute rules as to when 
and how to implement them in the instruction. But it is critical that teachers should use them flexibly at the 
appropriate time. 
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Abstract 

Four instructional principles for alleviating cognitive overload in online learning are suggested: 1 ) Guide 
learners to prepare and maintain an effective workstation for accessing online materials, 2) Employ advance 
organizers for effective online navigation, 3) Arrange instructional materials for easy online manipulation, and 4) 
Organize instruction to facilitate learners toward developing personal cognitive strategies for meaningful 
interpretation. This article concludes that these instructional principles should be empirically examined about their 
effects on online learning achievement. 



Introduction 

Advanced technologies such as the Internet and World Wide Web are changing the nature of online 
learning processes (Hartley, 1999; Hill, 1997; Land & Hannafin, 1996; Wolfe, 2001). Online learning processes 
involve more processes than that in traditional learning. For examples, recognizing hyperlinks, accessing electronic 
materials, and navigating online materials are the online learning processes that do not exist in the traditional 
learning. Also, it is noted that greater cognitive load is required for online than traditional learning (Britt & Gabrys, 
2001; Hill, 1997; Hill & Hannafin, 2001; Yang, 2001). The existing well-established and widely -adopted 
instructional principles are mainly for traditional classroom instruction (Gagne, Briggs, & Wager, 1992; Reigeluth, 
1983). Unfortunately, these principles are inappropriate for online learning because they seem fall short in 
addressing the additional online learning processes or the cognitive overload phenomenon. New instructional 
principles that take into consideration of these additional processes and can alleviate cognitive overload 
phenomenon are necessary for ensuring the effectiveness of online learning. 

Purpose 

The purpose of this paper is to suggest instructional principles for facilitating online learning. Instructional 
design for effective online learning should encompass features that can moderate the cognitive overload 
phenomenon in online learning. This paper will propose four instructional principles that can be embedded in 
instruction to reduce cognitive overload in online learning regardless of content or a specific population. They can 
be systematically employed in 100% or hybrid web-based instruction with synchronous or asynchronous 
communications. 



Instructional Principles for Online Learning 

The suggested four instructional principles for online learning are: 

1. Guide learners to prepare and maintain an effective workstation for accessing online materials 

2. Employ advance organizers for effective online navigation 

3. Arrange instructional materials for easy online manipulation 

4. Organize instruction to facilitate learners toward developing personal cognitive strategies for meaningful 
interpretation 

Principle 1: Guide Learners to Prepare and Maintain an Effective Workstation for Accessing 
Online Materials 

Instruction should first guide learners to set up or find an effective and stable workstation that meets 
technical requirements for online learning. Successful learners make efforts to arrange a functioning learning 
environment for study (Trawick & Corno, 1995) and instruction should guide learners to prepare an appropriate 
learning environment (Ley & Young, 2001). In addition, instruction should prepare learners for maintaining an 
effective workstation for online learning. Online learners often find themselves in a situation of solving technical 
problems, which becomes a serious problem especially for less experienced online learners (Althaus, 1997; 
Grantham & Vaske, 1985; Hiltz, 1993; Kooley, Kelsey, & Lindner, 2003). The need of supporting technical 
problem solving is widely recognized (Althaus, 1997; Garland, 1993). Instruction should provide information of 
technical help facilities within or outside the institute that learners can seek for help. 
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Principle 2: Employ Advance Organizers for Effective Online Navigation 

Instruction should provide advance organizers that clearly indicate the locations of pertinent documents to 
guide learners in searching online materials within a course website. The effectiveness of navigating nonlinear 
online materials can be enhanced by providing advance organizers at the beginning of a course website, such as 
providing principles for navigating hypertext (Shapiro, 1998), showing a map of the structure of hypertext 
(Jonassen, 1998), and presenting materials in a list format with less-step hyperlinks, or low link densities, to produce 
the best overall search results (Khan & Locatis, 1998). Once learners have a clear picture of the locations of pertinent 
documents or have the ability to accurately locate specific documents within course websites, the complicated 
navigation task (Eveland & Sharon, 2000; Niederhauser, Reynolds, Salmen, & Skolmoski, 2000; Shapiro, 1999) 
becomes easier and chance of getting lost in cyberspace (Conklin, 1987; Marchionini, 1988; McAlees, 1989; 

Nielsen, 1990) becomes lower. As a result, learners can easily initiate online learning activities with less cognitive 
resources in navigating online materials. 

Principle 3: Arrange Instructional Materials for Easy Online Manipulation 

Instruction should arrange materials for easy online manipulation and proper displays on computer screens. 
The amount of the materials displayed on a web page should either avoid being too long or be constrained within the 
display size. With properly arranged online materials for easy online manipulation, the construction of text -based 
mental model will require less cognitive load and the development of or shifting to online learning style will be 
smoother. As a result, cognitive resources required for manipulating online materials may be moderated. 

Principle 4: Organize Instruction to Facilitate Learners toward Developing Personal Cognitive Strategies for 
Meaningful Interpretation 

Instruction should incorporate cognitive strategies to guide learners in constructing meaningful 
interpretation from nonlinear online materials. Cognitive strategies are various methods that learners use to guide 
their own learning, thinking, acting, and feeling (Driscoll, 2000). The positive effects of employing cognitive 
strategies on constructing meaning from learning materials, such as showing a concept map of the whole text at the 
beginning is considered a good hypertext (Jonassen, 1998) and signaling the structure of text is viewed crucial to 
comprehending hypertext (Meyer, 1985). Guided by cognitive strategies, learners can gradually develop the ability 
to construct meaningful situation models for meaningful interpretations. Thus, the cognitive overload associated 
with constructing meaningful situational models can be alleviated. 

Conclusion 

This paper suggests four instructional principles for facilitating the online learning processes. These four 
principles should be able to alleviate the cognitive overload phenomenon in online learning. Future studies should 
empirically examine the effects of these four instructional principles on online learning achievement to assess their 
contribution to online learning and instruction. 



References 

Althaus, S. L. (1997). Computer- mediated communication in the university classroom; An experiment with 
on-line discussions. Communication Education, 46, 158-174. 

Britt, M. A., & Gabrys, G. L. (2001). Teaching advanced literacy skills for the World Wide Web. In C. R. Wolfe 
(Ed.), Learning and teaching on the World Wide Web (pp. 74-91). San Diego, Calif.; Academic Press. 

Conklin, J. (1987). Hypertext: An introduction and survey. Computer , 17-41. 

Driscoll, M. P. (2000). Psychology of learning for instruction . U.S.; Massachusetts. 

Eveland, W. P., Jr., & Sharon, D. (2000). Examining information processing on the World Wide Web using 
think aloud protocols. Media Psychology, 2, 219-244. 

Gagne, R., Briggs, L., & Wager, W. (1992). Principles of instructional design. Fort Worth, TX; Harcourt Brace 
Jovanovich. 

Garland, M. (1993). Student perceptions of the situational, institutional, dispositional and epistemological barriers to 
persistence. Distance Education, 14(2), 181-198. 

Grantham, C., & Vaske, J. (1985). Predicting the usage of an advanced communication technical. Behavior and 
Information Technical, 4, 327-335. 

Hartley, K. W. (1999). Media overload in instructional web pages and the impact on learning. Educational Media 
International, 36(2), 145-150. 



99 




Hill, J. (1997). The World Wide Web as a tool for information retrieval: An exploratory study of users' 
strategies in an open-ended system. School Library Media Quarterly, 25(4), 229-236. 

Hill, J., & Hannafin, M. (2001). Teaching and learning in digital environments: The resurgence of resource-based 
learning. Educational Technology Research and Development. 49(3), 37-52. 

Hiltz, S. (1993). Correlates of learning in a virtual classroom. International Journal of Man-Machine Studies, 39, 
71-98. 

Jonassen, D. H. (1998). Hypertext principles for text and courseware design. Educational Psychologist, 21(4), 269- 
292. 

Khan, K., & Locatis, C. (1998). Searching through cyberspace: The effects of link display and link density on 
information retrieval from hypertext on the World Wide Web. Journal of the American Society for 
Information Science, 49(2), 176-182. 

Kooley, K., Kelsey, K., & Lindner, J. (2003). Doc@distance immersion in advanced study and inquiry. 

Quarterly Review of Distance Education, 4( 1), 43-50. 

Land, S. M., & Hannafin, M. J. (1996). A conceptual framework for the development of theories -in -action with 
open-ended learning environments. Educational Technology Research and Development, 44(3), 37-53. 

Ley, K., & Young, D. B. (2001). Instructional principles for self-regulation. Educational Technology 
Research and Development, 49(2), 93-103. 

Marchionini, G. (1988). Hypermedia and learning: Freedom and chaos. Educational Technology, 8-12. 

McAlees, R. (1989). Navigation and browsing in hypertext. In R. McAleese (Ed.), Hypertext: Theory into Practice 
(pp. 10-19). Oxford: Intellect. 

Meyer, B. (1985). Signaling the structure of text. In D. H. Johassen (Ed.), The technology of text: Principles for 
structuring, designing, and displaying text (Vol. 2, pp. 64-89). 

Niederhauser, D. S., Reynolds, R. E., Salmen, D. J., & Skolmoski, P. (2000). The influence of cognitive load on 
learning from hypertext. Journal of Educational Computing Research, 23(3), 237-255. 

Nielsen, J. (1990). Hypertext and hypermedia. Boston: Academic Press. 

Reigeluth, C. (Ed.). (1983). Instructional-design theories and models: An overview of their current status New 
Jersey: Lawrence Erlbaum Associates. 

Shapiro, A. M. (1998). Promoting active learning: the role of system structure in learning from hypertext. Human- 
Computer Interaction, 13(1), 1-35. 

Shapiro, A. M. (1999). The relevance of hierarchies to learning biology from hypertext. Journal of the 
Learning Sciences, 8(2), 215-243. 

Trawick, L., & Corno, L. (1995). Expanding the volitional resources of urban community college students. New 
Directions for Teaching and Learning, 63, 57-70. 

Wolfe, C. R. (2001). Learning and Teaching on the World Wide Web. San Diego, California: Academic Press. 

Yang, S. (2001). Synergy of constructivism and hypermedia from three constructivist perspectives - social, 
semiotic, and cognitive. Journal of Educational Computing Research, 24(4), 321-361. 



100 




Adapting reading intervention for online students 

Baiyun Chen 
Atsusi Hirumi 

University of Central Florida 

Background Information 

With the advances of computer and network technologies, student enrollment in distance education has 
increased rapidly since the 1990’s. In 2003, it is estimated that about 40,000 to 50,000 K-12 students are enrolled in 
online courses nationwide (Golden, Wicks, & Williams, 2004). With the ever-increasing number of K-12 students 
who attend online program, researchers state in a report on virtual school in United States that “online education 
program(s) ... already are having a significant impact on public education” (Watson, Winograd, & Kalmon, May 
2004). Most likely, the K-12 online programs will take an increasingly important role in the school system of the 
United States. According to Building a Snapshot of Virtual Schools Across the Nation by Collins, “12 states have 
established online high school programs and 5 others are developing them... 25 states allow for the creation of so- 
called cyber charter schools, and 32 states have e-learning initiatives under way” (Collins, 2004). In the future, the 
option of e-learning will be available to every child for purposes of advanced study, credit recovery or remedial 
learning. 

The target population of a virtual school encompasses gifted students, students seeking credit recovery, and 
at-risk and dropout students. The focus of this paper is the problematic students who do not succeed in traditional 
classroom. The at-risk and dropout student population size is shockingly large. In 2000, some 3.8 million young 
adults were out of school without a high school credential, accounting for 1 1% of 16- to 24-year-olds in the United 
States (Kaufman, Alt, & MPR Associates, 2001). In addition, there were 612,900 students, or 1.3% of all public 
school students enrolled in public alternative schools or programs for at-risk students in 2000 (Kleiner, Porch, & 
Farris, 2002). Considering the big proportion of at-risk and dropout students, educators should seek effective 
methods to help them finish their secondary school education. 

For students who fail in regular secondary school programs, online learning emerges as either a supplement 
or replacement for face-to-face classroom instruction. The benefits of attending an online program are its flexible 
accessibility, individual paces, assistive resources and absence of social label. In an online environment, the students 
could log into class at a flexible schedule. The Internet allows students and instructors to access the network and 
teach from anywhere and at anytime where there is an Internet connection and a multimedia computer. Students 
could revisit course materials whenever they need and pace the learning progress at their comforts. Moreover, there 
is abundance of study aid resources available online, such as e-dictionary, audio and video elements, and other 
multimedia technologies, which usually results in a higher learner motivation. Last but not least, the at-risk and 
dropout students will not feel labeled as less capable during their online learning process. Their desire to become a 
part of a group and be accepted is fulfilled. 

While a large number of at-risk and dropout students are electing the online option, it is the educators’ 
responsibility to produce and deliver effective online learning experience. The research topic is narrowed to reading, 
because one primary reason for school failure at each behavior is due to reading problems. The struggling children 
are challenged by decoding and encoding problems, limited word -recognition ability, poor metacognitive skills, or 
lack of reading comprehension strategies (O Brien, 2001). On the other hand, in a complexly networked information 
environment the abilities to read become even more important when reading and writing is the essential form of 
communication, rather than listening or speaking. Furthermore, reading will “take new forms as text is combined 
with new media resources and linked within complex information networks” (Leu, 2002), which requires new 
reading comprehension methods. However, the reading intervention and assessment are still mostly focused on 
outcome measure from traditional paper and book text, not presenting or evaluating new literacies of this 
information century. All the above stated are problems that prompt this research. The primary purpose is to explore 
and elaborate the above questions and to develop and test an online reading intervention module for problematic 
secondary students to improve their reading abilities. 

Comprehension Strategies 

To construct a reading intervention module online, research has been done on reading skills in traditional 
classroom context. According to the National Reading Panel Report in 2000, reading instruction is effective in five 
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areas: phonemic awareness, phonics, fluency, vocabulary, and text comprehension. However, the emphasis on the 
five reading skills is different among students of different ages. Generally, the basic reading ability components, 
such as phonemic and phonics awareness, are essential for early childhood reading interventions. As the children 
grow older, the emphasis of their reading intervention would shift to the higher-order skills, such as fluency, 
vocabulary, and comprehension. For secondary school students, more specifically, comprehension is important. 
Many research findings reveal that poor readers improve their text comprehension by learning to use comprehension 
strategies (Armbruster, Lehr, & Osborn, 2001; Mastropieri, 2003; Swanson, 1999; Williams, 2002). Armbruster 
defines comprehension strategies as “conscious plans and sets of steps that good readers use to make sense of text.” 
Direct evidence shows that “comprehension strategy instruction helps student become purposeful, active readers 
who are in control of their own reading comprehension”(Armbruster et ah, 2001). 

Based on previous researches, the following eight comprehension strategies have a firm scientific basis for 
improving text comprehension. They are respectively a. Activating prior knowledge; b. Recognizing text structure; 
c. Constructing visual representations; d. Drawing inferences; e. Summarizing; f. Generating questions; g. Thinking 
aloud; and h. Monitoring and repairing comprehension. 

Good readers recall prior experience and information relating to topic to help them understand what they 
are reading. Research findings indicate that students benefit from prior knowledge about the form and organization 
of the content (Spires, Gallini, & Riggsbee, 1992) and the background knowledge measure is a significant and 
reliable predictor of passage-specific comprehension (O. S. Anderson & Acker, 1984; Langer, 1984). When a 
student has activated prior knowledge, the student is better able to focus on what is important in the text. 

Recognizing text structure is another proved effective comprehension strategy. Often, students learn to 
attend to and uncover text organization through the use of story maps. Researches show that students who recognize 
the text structure have the greater appreciation, understanding, and memory of the text (Armbruster & Anderson, 
1984; Armbruster et ah, 2001), and instruction in text content and organization improves students comprehension 
and memory (J.F. Baumann & Bergeron, 1993; Gersten, 2001; Lorna Idol, 1987; L. Idol & Croll, 1987). 

The third strategy is forming visual representations to illustrate concepts and interrelationships among 
concepts in texts. Proficient readers use mental images to deepen their understanding of the text and solve problems 
(Rose, Cundick, & Higbee, 1983). Research findings revealed that instructions to form mental imagery, given prior 
to reading a text, increased literal comprehension and monitoring skills (Chan, 1990; Gambrell & Koskinen, 1982). 

Drawing inferences is the process that is involved as students make predictions before and during reading. 
This process includes judging, concluding, or reasoning from given information. It has been described by some 
researchers as the heart of the reading process (R. C. Anderson & Pearson, 1984). Researchers have found that 
readers improve their abilities to construct meaning when they are taught how to make inferences (Hansen, 1981; 
Hansen & Pearson, 1983; Raphael & Wonnacott, 1985). 

A summary is a synthesis of the important ideas in a text (Armbruster et ah, 2001). To summarize a reading 
text, students are required to determine what is important in text, to eliminate redundant and unnecessary 
information, and to condense the main ideas into their own words. Summarizing has been shown to be an important 
strategy in help readers improve their abilities to construct meaning and writing (Taylor & Beach, 1984). 

Teaching students to ask questions improves their active processing of text as well as comprehension. By 
generating questions, students learn to ask themselves questions that require them to integrate information from 
different segments of text (Armbruster et ah, 2001). Brown and Palincsar (Brown & Palincsar, 1982) and other 
researches (Andre & Anderson, 1979; Buehl, 2001; Cohen, 1983) demonstrated how effective student-generated 
questions can be in helping students to improve their abilities to construct meaning and to motivate reading interests. 

Students thinking aloud has also been shown to increase comprehension (J.F. Baumann, Jones, & Seifert - 
Kessell, 1993; James F. Baumann, Seifert-Kessell, & Jones, 1992; Davey, 1983; Oster, 2001). This strategy requires 
a reader to verbalize his/her thoughts as they read. While they stop periodically in reading, they spend time 
reflecting on how a text is being processed and understood. 

Monitoring, the process of knowing when what you are reading is not making sense and having some 
means for overcoming the problem, is an important part of students’ metacognitive development. Successful 
learners monitor their own comprehension and adjust their learning strategies accordingly (Brown & Palincsar, 

1982; Paris, Lipson, & Wixson, 1983). Strategies for monitoring include asking oneself if the reading is making 
sense, rereading, reading ahead, looking up words in the dictionary, guessing word meaning, or asking someone for 
assistance. 

In addition to identifying which comprehension strategies are effective, scientific research provides 
guidelines for how to teach these comprehension strategies. Research findings indicates that effective 
comprehension strategy instruction is explicit or direct (Dole, 2000; Duffy, 2002; Hancock, 1999; Mastropieri, 

2003; Swanson, 1999). When teaching comprehension strategies, teachers tell readers why and when they should 
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use strategies, what strategies to use, and how to apply them in explicit language. The components of explicit 
instruction typically include direct explanation, teacher modeling (“thinking aloud”), guided practice, and 
application (Armbruster et ah, 2001) or independent use of strategies. 

Recommended Intervention & Conclusion 

Recommended Intervention 

In the secondary online reading remedial program, students need to be taught to become strategic readers 
by learning the effective comprehension strategies. How should we teach that in the online environment? Shall we 
copy every step we have had in face-to-face classroom in online learning modules? What changes we should make 
in web-based reading intervention? Since distance online education is different from traditional school 
environment(Dzuiban, Shea, & Arbaugh, 2004; Easton, 2003), students are no longer listening to a lecture with eye 
contact and gestures from the teacher. The online learning unit needs to be redesigned, and a new instructional 
strategy needs to be adopted(Dzuiban et ah, 2004; Easton, 2003). Effective instructional steps in face-to-face 
classroom could be adapted in the online context while network and technology resources are integrated to 
compensate for the loss of face-to-face interactions. 

The proposed online strategy instruction module teaches students one of the above comprehension 
strategies, “Activating Prior Knowledge”. This web module encompasses elements from student engagement, 
explicit explanation, teacher modeling, and learning outcome evaluation. All the necessary components of explicit 
instruction are integrated within the designed online strategy. The module script is attached in the appendix. 

At the beginning of the course, students are welcomed by instructor and given a direct explanation why the 
strategy helps comprehension and when to apply the strategy. The technology media chosen in this part include text, 
graphic, or multimedia illustration, if needed. Different media are used to motivate students so that they are not 
turned away by boring preaching. 

Refer to Figure One 

After students are taught of why and how the strategies are to be used, a video link is provided in which 
either a teacher or a student models or demonstrates how to apply “Activating Prior Knowledge”. In this video, the 
teacher would verbalize their thoughts while reading a text that the students are using. The teacher would model the 
three types of connections that students can make between text and their own knowledge and experience separately. 
In this way, it is easier for students to copy the teacher’s instruction and transfer the strategy into their own reading 
process. 

Refer to Figure Two 

After watching the strategy modeling video, students proceed to a practice test at the guidance of their 
parents or mentor. Students would download an activity sheet and practice orally using this strategy with parents or 
mentor and fill in an activity form. In the designed activity, students are encouraged to read one small piece of news 
from an online kid magazine. They are instructed to link their background knowledge before, during and after 
reading process respectively. It is important the students are doing this practice with their parents or mentor, so that 
they might be offered help or monitored throughout the process. 

Refer to Figure Three 

The fourth step is one student activity which is supported by one type of distance communication tools, 
such as bulletin board discussion, online chatting room, telephone conferencing or even telephone calls. Now 
students would write down what they have practiced orally and post their thoughts in the discussion area. In this 
way, learners could read other students’ postings and learn from how to make connections and how to apply the 
strategy. The collaborative activity is designed here to help build a learning community between students to enhance 
learner- learner interactions, thus promoting higher learning motivations. 

Refer to Figure Four 

The last component of the designed module is evaluation. This is comprised of a quiz to test students’ 
verbal knowledge of strategy definition and guidelines and a reading assignment for them to apply the strategy 
independently. The quiz takes format of true/false, multiple choice and short answer questions, to evaluate students’ 
abilities to identify why, when and how the strategies are carried on in reading process. In the reading assignment, 
they are instructed to read one chapter of a story book and write a three-paragraph assignment using “Activating 
Prior Knowledge”. A detailed rubric is given so that students could follow certain format to finish their assignment. 
Refer to Figure Five 
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Conclusion 

To help secondary school at-risk or dropout students, reading is an indispensable instructional area. In this 
paper, the importance of comprehension strategies instruction is stressed as one integral part of online reading 
remedial program. At the same time, this paper also points out some remaining challenges for educators and 
researchers of distance education. Major areas are now discussed. 

1. Comprehension strategies help low-performance students to become strategic readers. In 
distance learning where reading and writing are the main methods for information 
communication, disabled students need to learn these strategies to construct meaning in all 
content subject areas. 

2. Students can be taught to use comprehension strategies through explicit instruction. The five 
steps of explicit instruction can be adapted in online setting, with integration of computer and 
network components, such as web pages with text and graphics, bulletin discussion, and audio 
or video elements to guarantee a high-quality interactive online learning experience. 

3. The online reading module of teaching “Activating Prior Knowledge” has been designed and 
developed based on an e-learning instructional strategy integrating the effective traditional 
instructional approach and well-selected technical media. Further experimental research needs 
to proceed to test its effectiveness and validity. 

4. More reading modules on teaching other reading skills, such as phonemic, phonic, fluency or 
vocabulary either in a separate or integrated fashion, should be developed and tested in future 
for facilitating reading skills for secondary -level low-performance students. 

5. Besides the eight evidence-proven effective comprehension strategies, navigation is identified 
as another essential strategy for reading online. This is a unique but important information 
searching and evaluation method in online learning process. Recent studies call attention to 
such skills of locating and analyzing web information (Leu, 2002; Schmar-Dobler, 2003) in 
new networked information literacies. However, further empirical evidence is needed to prove 
its value on students’ comprehension during their webpage reading. Research should be 
conducted to support its importance in distance learning process and to provide guidelines for 
how to teach this strategy. 
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Appendix 
Module Transcript 

Module One 
Parti 

Course Overview 

Do you understand the information you read in your textbooks, stories or newspapers? One of my former students 
told me that she tried to understand the text used but she did not really comprehend what she read. Does this sound 
like you? In this class, we are going to learn a few strategies that you could use while reading to make sure that you 
understand the meaning of stories and information you read. When you master these simple strategies, you should 
find reading an easy pleasure and not at all simply homework that teachers have assigned. 

Module Overview 
Time limit Two 45-minute class 
What we are going to learn? 

1. What is “Activating Prior Knowledge”? 

2. Why and when to use the strategy? 

3. What are the connections we can make? 

4. How do we use the strategy? 

The first strategy that we are going to learn is “Activating Prior Knowledge.” This is a simple task. Everybody has 
prior or background knowledge. This includes a sporting experience you had with your friends, or stories that your 
grandparents told you when you were younger. 

Why do we want to think of prior knowledge while reading? Good readers do this because having previous 
knowledge helps us build a stronger interest for different reading topics, remember important ideas, organize new 
information, and add to our learning and understanding of new and interesting subjects. 

When do we activate prior knowledge? We should do this before, during and after reading. Before reading a text, 
look at the books title or skim the first paragraph. You might already have some experience or knowledge related to 
the topic. During the reading, you might read sentences that remind you of some past experiences . Is your 
experience the same with what the text tells you? After reading, you close the book and compare what you have 
learned from this text with what you have thought before reading. Are they the same story? Or what changes you 
have now on this topic? 

Module One 
Part II 

Three Connections 

After explaining the reasons and the time we should activate prior knowledge. You might have the question: “What 
prior knowledge I could have? I don’t know this topic.” To make the activating process easier, we will look at the 
three types of connections that you could refer to. These connections are: 

Text- to -text - connections between different books and different authors 

Example : Have you ever heard of J.R.R Tolkien? Have you ever seen Lord of the Rings? Did you know Tolkien 
wrote Lord of the Rings? 

This is a fable. Have I ever read a fable? 

Text- to -me - connections between books and the readers current personal background knowledge, experience, and 
individual interests and feelings. 

Example: This is a story of the Olympic Games. I like sports very much. I want to become a professional basketball 
player when I grow up. The title is “Volcano.” I saw a volcano once on a trip to Hawaii with my parents. 

Text- to -world- connections between books and information about the world around us. 

Example: This is about the people who had AIDS. I’ve seen many television shows about this disease. 

The boy in the story is Indian. I don’t know much about India. Maybe it is in Asia, very far away? 

How do we use the strategy? 

How do I activate prior knowledge? Here is a video clip where a teacher is modeling the “activating prior 
knowledge” strategy for you. Look at what the teacher is doing and follow the same steps during your reading. 

You will need Quicktime installed in your computer to play this video. If you don’t have this program, follow this 
download link to download and install the software. Please get help from your parents or mentor if you have trouble. 
View Strategy Modeling Video 
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Module One 

Part 111 

Guided Practice 

Now it’s your turn to do a practice with your parents or mentor. If you don’t have a person to help you practice, 
please contact your instructor and we can help you over the telephone. Download the two files. This first one is a 
reading paragraph. The second one is a checklist that you need to complete with your mentor. 

This is the news that you will read. If you have difficulty in locating the news, please download this plain text 
version . 

Pets Get Their Rights in Italy 

By Sharon Thompson 

National Geographic Kids Magazine 

September 15, 2004 

People in Reggio Emilia, a town in central Italy, must pamper their pets or pay a fine of up to 
500 euros (about U.S. $600) according to a new law. 

Dog owners are required to provide roomy doghouses, and owners of a single canary or parakeet 
must buy a second bird so their pets won't get lonely, according to the law. 

And you won't find pets sporting racing stripes in Reggio Emilia. Citizens there may no longer 
dye their pets' fur. 

This is the activity you will do with your parents or mentor. 

Activating Prior Knowledge 
Making Connections Activity 

Follow these steps with your parents or mentor. 

Before reading 

Step 1: Read the title and the first sentence 

Step 2: Tell your parents or mentor if the topic reminds you of your own story? 

Step 3: If you do, write down your thoughts 



Choose the connection type you make 


Your story 


( )Text-to-text connections 




( )Text-to-me connections 


( )Text-to-the-world connections 



During reading 

Step 1: Begin to read the paragraph 

Step 2: Tell your parents or mentor if you connect the topic with your prior knowledge or experience? 
Step 3: If you do make connections, write down your thoughts 



Choose the connection type you make 


Sentence # 


Your story 


( )Text-to-text connections 






( )Text-to-me connections 


( )Text-to-the-world connections 



After reading 

Step 1 : Review your own story 

Step 2: Tell your parents if it is different from what you read from this paragraph? 



Discussion Posting 

Now let’s see what you get from this activity. Write down one of the stories that you get from this activity and post 
it in discussion area. Before posting your story, read the rubric to see if you have done this correct. 

Rubric for Discussion Posting 

Here is what I am looking for in you discussion posting. 

I =1 n ad equate 2= Average 3=Outstanding 

(Assignment needs to be redone) 



1 


Show which type of connections you make in your paragraph. 


1 


2 


3 


2 


Indicate when you make the connection, before or during 
reading. 


1 


2 


3 


3 


Point out which part of the paragraph reminds you of your own 


1 


2 


3 
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story. 








4 


Your story is described using at least three sentences. 


1 


2 


3 


5 


The spelling, punctuation, and grammar are accurate. 


1 


2 


3 


6 


The posting is under the right topic in discussion area. 


1 


2 


3 



Module One 

PartlV 



Quiz 

You have succeeded in activating your prior knowledge. Let’s refresh our memory and see if you still remember 
why, when and how this strategy is used by taking a short quiz. You have two chances to take it. After each quiz, 
you will get your results in your email box. 

Take the quiz on “Activating Prior Knowledge”. 

Quiz I 

1. Connecting a story with our own experience and knowledge helps us have a better interest in reading 
certain books. 

a. True b. False 

2. You can only activate prior knowledge before you read a book, 

a. True b. False 

3. “When I read a fable, I remember the times my mom used to read bedtime stories to help me sleep.” Which 
type of connection is this? 

a. Text-to-text 

b. Text -to- me 

c. Text-to-world 

d. Text-to-knowledge 

4. “This is a story about a musician in Austria. I am interested in it, because I am practicing piano right now. I 
want to become a professional pianist in the future.” Which type of connection is this? 

a. Text-to-text 

b. Text -to- me 

c. Text-to-the-world 

d. Text-to-knowledge 

5. We are going to the zoo. Can you name three animal names in a zoo? 



6. We are going to read a story of Olympic Games. What do you know about this topic? 



Reading Assignment 

You are now an expert in using “Activating Prior Knowledge” in reading! Let’s try to read a whole chapter of one 
book. While you are reading, be sure to apply the strategies you have learned and make connections between the text 
and your story. You will need a pen and a piece of paper to write down your thoughts. After finishing the reading 
you will write a three -paragraph paper for this class. Read the rubric and check if you have written the correct piece. 
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Rubric for Reading Assignment 



Here is what I am looking for in you writing for this reading assignment. 

I ^Inadequate 2= Average 3=Outstanding 

(Assignment needs to be redone) 



1 


Write 3 paragraphs for this assignment. 


1 


2 


3 


2 


In the 1st paragraph, show which type of connections you make 
in your reading. 


1 


2 


3 


3 


In the 1st paragraph, show when you make the connection, 
before or during reading. 


1 


2 


3 


4 


In the 2nd paragraph, point out which part of the story reminds 
you of your own story, the title or a particular paragraph. 


1 


2 


3 


5 


In the 2nd paragraph, write one piece of your own experience or 
knowledge you think of in reading. 


1 


2 


3 


6 


In the 3 rcl paragraph, compare your own story with what you get 
from this book. 


1 


2 


3 


7 


The spelling, punctuation, and grammar are accurate. 


1 


2 


3 



Figure 1: An explicit description of the strategy and why and when it should be used. 
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Figure 2: Teacher and/or student modeling of how the strategy is used in action. 
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Figure 3: Guided practice using the strategy with gradual release of responsibility. 
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Figure 4: Collaborative use of the strategy in action. 
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Interactive TV : An Effective Instructional Mode for Adult Learners 



Li- Ling Chen 
Carole Iris 

California State University, Hayward 

The inclusion of interactive television (iTV) programs for learning is an emerging genre in education. 
Literature has concluded that any aspect of learning requires some form of interaction or feedback to be most 
effective. As television (TV) evolves from being a passive to an active medium, it has the potential to engage 
learners and reach a mass audience on a scale much larger than traditional education and training. 

The term digital TV refers to the distribution and transmission of audio and video digitally to its 
destination. Interactive digital TV (iTV) means that control moves away from the broadcaster and network and is 
placed directly in the hands of the potential consumer. ITV programs are TV programs with new forms of engaging 
educational content enhanced with interactivity. In this paper, iTV instruction refers to any learning utilizing iTV 
programs. iTV learning is a type of learning that utilizes interactive TV or similar screen-based devices. Video-rich 
content is delivered via one or a number of different platforms. iTV is not learning with a personal computer as e- 
learning is recognized today. 

Adult learners are typically considered to be people older than the age required for compulsory school 
attendance. The changing life course of adults is closely related to lifelong learning. Demographers conceptualize 
the life course as the regularized patterns of transitions and roles that individuals experience as they age. The life 
course norm has shifted from generally routine and predictable transitions (e.g., from student to worker, from spouse 
to parent) to one in which lives are more “disorderly” and people hold a variety of roles (Lifelong Learning NCES 
Task Force, p. 59; Rindfuss, 1991). 

The purpose of this paper is twofold: to identify why iTV is an effective instmctional mode for adult 
learners and to promote the integration of iTV instruction into lifelong learning and adult education. The popularity 
of iTV in Europe and the potential of iTV in the United States is introduced first. The theoretical framework that 
supports the effects of iTV for learning is discussed. Identified later are the reasons why adults want to learn and the 
forces that influence adults to learn. The potential and benefits of iTV instruction for adult learners will also be 
addressed. The challenges of such instruction are identified. Finally, conclusions on the importance and potential of 
iTV instruction for future adult education are presented. 

Popularity of Interactive TV 

Interactive TV has a strong foothold in the European TV market. While in the past few years, US (United 
States) audiences were just beginning to experience basic interactive TV services. Comparative figures for the end 
of 2001 showed that 37% of UK (United Kingdom) households had digital TV compared with a European average 
of 16.3%. By 2003 the figure had increased to nearly 44% of British households (Pastore, 2002). 

Interactive TV in the U.S. is set to grow 83 percent per year through 2005 compared to online growth that 
is predicted to grow only at nine percent during the same period. Interactive TV will be an accelerated growth 
market, reaching approximately 46 million homes by 2005. It is predicted that the U.S. will outpace Europe as a 
whole in interactive TV penetration by 2005 (Waiting Critical Mass, 2001). Jupiter Media Matrix forecasts that 72 
percent of digital cable households and 77 percent of digital satellite households will be interactive by 2007 
( Loizides , Gartenberg, & Peach, 2002). By 2008, TV households for digital TV and interactive TV markets for pay 
will grow to 69 million and 54 million, respectively ( Loizides , Card, & Patel, 2003). It is also predicted that by 
2005, 625 million people worldwide would use services offered by digital TV (Strategy Analytics Research, 2001). 

Theoretical Framewo rk for Supporting iTV Instruction 

The major theoretical framework to support the effects of iTV for instruction is the interactivity theory 
proposed by Vaughan (1998). He proposed that interactivity empowers the end-user of the application by letting 
them control the content and flow of information. Interactivity permits the user to navigate and explore the 
application at his or her own pace. This is an important feature in a learning tool in which a learner can view and 
study at his or her own speed and skill level, thus controlling the pace of learning. 

Research has shown that active learning occurs when a learner engages three cognitive processes: (1) 
selecting relevant words for verbal processing and relevant images for visual processing; (2) organizing words into a 
coherent verbal model and images into a coherent visual model; and (3) integrating corresponding components of 
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the verbal and visual models (Moreno & Mayer, 2000). Moreno and Mayer stress that simply presenting verbal 
explanation of how a system works with an animation does not insure that learners will understand the explanation. 
Learners have to be able to cognitively engage in the learning process by choosing, organizing and integrating their 
learning materials. iTV instruction allows learners to select learning materials, to organize and to integrate the 
selected content. Based on this perspective, iTV is a perfect tool; it provides an active and meaningful way for 
learners to engage in his/her learning. 

Forces Influencing Adults to Learn 

One of the main reasons adults seek learning experiences is to help them cope with specific life -changing 
events; transitions such as marriage, divorce, a new job, retirement, lose of a loved one or a move to a new location. 
As stress increases and accumulates with life-changing events the adult’s motivation to engage in learning 
experiences increases. Increasing or maintaining a sense of self-esteem and pleasure are strong motivators for adults 
to engage in learning experiences (Zernke, 1984). Adults' readiness to learn is also directly linked to need— needs 
related to fulfilling their roles as workers, spouses, parents, etc. 

According to Hiemstra (2003), there are three major forces that act in concert to generate this interest and 
need for adults to learn. The first of these can be described simply as the rapidity and constancy of change in the 
society. A second major force is the continuous march by many adults toward occupational obsolescence. Adults 
frequently must turn to learning activities in and out of the workplace just to maintain or regain competence. The 
third force that has helped create the interest in, and need for, lifelong learning deals with the change in lifestyles or 
value systems affecting so many people. 

Friedrich (1993) emphasized the importance of self -directed learning in adults. This self-directed type of 
learning appears to be related to life satisfaction and independence in adults. He also claimed that adults want to 
learn because “humans are unfinished beings; they are in the process of becoming. This unfinished character of 
humans along with the changing character of reality, requires that education be an ongoing activity. Therefore, 
people have not finished their education when they reach old age; rather it is a lifelong process.” 

Potential of iTV Instruction for Adult Learners 

Interactive TV holds promise for adult learners based on following three reasons: iTV instruction can 
motivate adult learners, iTV instruction promotes adult learners’ self-efficacy and iTV instruction offers an 
environment conducive for learning. The specific discussion follows. 



iTV Instruction Can Motivate Adult Learners. 

Few adult education textbook authors devote any attention to the role of motivation in learning or the 
developmental course of adult motivation (Smith & Gallagher, 2002). Adults are more often internally motivated by 
the potential for feelings of worth, self-esteem, achievement, etc. Pourchot (1999) has suggested that the 
development of a sense of generativity in middle adulthood is positively associated with increased intrinsic 
motivation. As adults develop, they are therefore likely to experience changes in how and where their motivation is 
directed. Adult educators should assist adults in appropriately channeling their motivation in order to accomplish 
their personal learning goals. 

The best educational TV and films use imagery to captivate and stimulate the audience. Those skilled in 
this medium know how to tell a story, how to intrigue and convey ideas effectively. In contrast, developers of 
adaptive educational software have focused on individualizing the content and its presentation to motivate the user. 
Interactive TV programming can motivate adult learners because it requires the innovative merging of these two 
approaches (Luckin, et Ah, 2003). 

Recreating a semblance of a traditional teaching environment (classroom) for the interactive TV interface is 
but one possible initial scenario that may be used to build user familiarity with this new medium of instmction. 
Interactive TV offers possibilities for an innovative learning environment 

with the ability to overcome the physical and temporal limitations imposed by the traditional educational process. 

By its intrinsic nature, it enables the creation of a rich, dynamic and stimulating “virtual environment” which 
constitutes a new learning context that directly influences perception, activation and memory and develops a new 
way of thinking (Garito, 2001). 

iTV Instruction Promotes Adult Learners’ Self-Efficacy. 
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According to Bandura (1982), self-efficacy is a self judgment of one’s ability to perform a task within a 
specific domain. Self-efficacy influences the choices people make, the effort that is put forth and the level of 
persistence when confronted with obstacles. How people behave can often be better predicted by the beliefs they 
hold about their own capabilities than by what they are actually capable of accomplishing, for these self-perceptions, 
which he called. . .self-efficacy beliefs, help determine what individuals do with the knowledge and skills that they 
have (Pajares, 2002). 

Research studies concluded that adult learners’ self-efficacy scales were correlated with media uses and 
exposure (Hofstetter, Zuniga, & Dozier, 2000). Media self-efficacy exists in adult learners. iTV instruction uses 
television as a tool for instruction that assists the promotion of adult learners’ self-efficacy. 

iTV Instruction Offers a Conducive Learning Environment. 

Setting a climate that is conducive to learning is vital. There are at least two aspects of climate: physical 
environment and psychological atmosphere. The psychological climate is even more important than the physical 
climate according to Rossman (2000) who states, “Adults must feel secure and safe within any learning situation. 
They must be treated with respect and dignity. Learners must also feel supported.” Knowles (1989) wrote, "People 
learn better when they feel supported rather than judged or threatened." 

Television is not a new technological tool for adult learners. Interactive TV instruction based on the 
technology and concept of television provides adult learners a learning environment that is familiar, comfortable, 
and conducive to learning. This type of instruction opens up the possibility of personalized, adaptive learning 
experiences for individual and group learners. 

Challenges of iTV Programming for Lifelong Learning 

The major challenges involved in the integration of iTV instruction for lifelong learning are the availability 
of iTV programs and the unstable iTV policy. There are very few appropriate iTV programs available for adult 
learners as iTV is presently an emerging technology for learning. Therefore, there is a great need for the design and 
development of educational iTV programs. When designing an iTV program for adult learners, it is very important 
that the general characteristics of adult learners be considered. Cave, LaMaster, & White (1998) outlined a list of 
adult learners’ general characteristics that presents important basic criteria to consider when designing or choosing 
appropriate educational programs for adult learners: 

1. Adults perceive themselves to be doers; and they prefer using previous learning to achieve success as 
workers, parents, and so on, and they have a broad, rich experience base to which to relate new learning. 

2. Adults depend upon themselves for material support and life management. They are largely self-directed, 
and they learn best when they perceive the outcomes of the learning process as valuable contributing to 
their own development, work success, etc. 

3. Adults are very different from each other. Adult learning groups are likely to be composed of persons of 
many different ages, backgrounds, education levels, etc. 

4. Adults, in addition to perceiving time itself differently than children do, are more concerned about the 
effective use of time. Although, for the most part, they learn more slowly than children, but they learn just 
as well. 

5. Adults are much more likely to reject or explain away new information that contradicts their beliefs. 

6. Adults' readiness to learn is more directly linked to need-needs related to fulfilling their roles as workers, 
spouses, parents, etc. and coping with life changes (divorce, death of a loved one, retirement, etc.). 

7. Adults are more concerned about the immediate applicability of learning. 

8. Adults have well -formed expectations, which, unfortunately, are sometimes negative because they are 
based upon unpleasant past formal learning experiences. 

Coupled with individualized content presentation and conveying ideas effectively, interactive TV 
programming is presented with the challenge of providing for the traditional social nature of television watching. It 
is quite probable that interactive lifelong learning programming will be used by small groups of users as well as solo 
users. Furthermore a formidable question arises, how can interactive TV programming adapt to a group of viewers, 
in such a way that each individual benefits from the material? This is a very important consideration, as it is known 
that even amongst adults of the same age significant differences will occur with respect to interests, knowledge and 
learning capabilities (Masthoff, 2002). 

Policy is a challenge as well as a factor that explains an important reason why US digital interactive TV 
currently lags behind Europe. The DVB (digital video broadcasting) project, an international consortium of 
broadcasters, manufacturers and regulatory agencies that began working in 1993 to establish a European standard 
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for digital transmissions has been highly influential. While the US standards committee got bogged down in political 
squabbles and technical issues, the DVB group developed a standard so technologically reliable that some US 
station owners have petitioned the FCC (Federal Communications Commission) to let them use it instead of the 
standard they agreed to (Rose, 2000). To promote the use of iTV and to integrate iTV technology for adult learners, 
the FCC in the US needs to establish digital transmission standards that are agreed upon and accepted by the 
television industry and relevant institutions. 



Conclusion 

Interactive TV instruction is an effective tool for providing adult learners new learning opportunities. In 
our rapidly changing society, more and more adults need to keep learning in order to cope with specific life - 
changing events or environments. Interactive TV instruction holds significant promise for adult learners in three 
potential benefits: iTV instruction can motivate adult learners, iTV instruction promotes adult learners’ self-efficacy 
and iTV instruction offers a conducive learning environment. 

Interactive TV instruction has tremendous potential to provide an optimal experience for the self -directed 
informal lifelong learner. Statistical information demonstrates that digital interactive TV is rapidly gaining a place in 
households worldwide, thus providing an excellent means of access to lifelong learning for many millions of adult 
learners who would otherwise be excluded from informal educational opportunity. 
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Building an online learning community 



Yu-Chien Chen 
University of Washington 

Introduction 

The Internet was not invented for education at beginning (Pett Grabinger, 1995), but it has influenced 
educational systems considerably, especially by providing another way for distance learning. This powerful 
communication function is superior to any other educational media. Students can conduct their own self-directed 
learning without interacting with others in an online learning environment. However, several scholars asserted that 
interaction would increase the learning quality in online learning. People who advocate Constructivism also claim 
that knowledge is built through interaction with environment, including materials and people (Jonassen, Peck & 
Wilson, 1999). Moreover, because of physical separation, it is more important to have social support due to the fact 
that we are human beings, and societal animals (Winn, 1997). In recent years, the MIT Media Lab 
( http://persona.www.media.mit.edu/SMG) has been devoted to develop online tools which will facilitate online 
communication to eliminate the “sociable spaces” (Karahalios & Donath, 2003). Therefore, the current trend of 
studying online learning has developed into the study of online learning interaction (Hung & Chen, 2001; Hung & 
Nichani, 200; Tu & Mclsaac, 2002; Conrad, 2002), switching from technical to social aspect. 

Along with the development of the Internet, some virtual communities have grown up, too. These virtual 
communities are groups where people share common interests (Rheingold, 2000) and discuss through computer- 
mediated communication (CMC). Some virtual communities survived and still operate well, while some of them 
disappeared. What kind of characteristics do these survival communities have? What contributes to their success? 
Online learning community is also one kind of virtual communities. The purpose of this paper is to discoverthe 
feature of virtual communities and propose some suggestions from experiences of successful virtual communities to 
building learning communities. The research questions are as follows: 

1. What characteristics do successful virtual communities have? 

2. How can we apply those experiences to enhancing online learning communities? 

Literature Review 

To begin with the literature review, the features of virtual communities will be discussed. Then examples 
of virtual communities will be examined. Last, some literature about how to build an online learning community 
will be analyzed. 

Virtual community 

In the past, the definition of “community” often contains three elements: the common interest of its 
members, linkage or interaction, and common location (Hillary, 1955). However, a community constructed on the 
Net is not restricted by physical space anymore. It is a community which is more librated from transportation and 
telecommunication (Wellman, 1979). In a virtual community, people can even be liberated from their daily lives. 
But, can we say that once we log into the Net, we enter a virtual community? A virtual community is a “social 
aggregation” in which amount of people participate in discussion for sustained periods, thus constituting 
interpersonal networks (Rheingold, 2000). Although a virtual community is not limited by physical boundary 
anymore, it still exists “the sense of place” (Coate, 1997). Only then, will the appearance of community be fostered. 

In a virtual community, the communication pattern is based on text. People there can communicate 
synchronously and asynchronously. Synchrony means one can communicate with one or many people in real time , 
such as by using MSN messenger or ICQ. On the other hand, in an asynchronous situation, like E-mail, there might 
be no simultaneous response. Even so, people can still share information and it can be called “ the compressibility 
of time and space” (Lin, 1996) in an asynchronous environment. In addition, people can not express moods or 
emotions by only using text . Some emotional expression symbols or graphs were created to help people express 
themselves. Another feature of virtual community is that in an online environment, people are anonymous and there 
is no face-to-face contact with others. In this situation, people can have more privacy and be encouraged to express 
themselves more freely. However, it does not mean that they do not have any chance to meet with each other (Chen 
& Lai, 1996; Rheingold, 2000) and this kind of face-to-face meeting plays a great role in online interaction. One 
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member of the online group TaNet Elephant-fan Association (TEA), said, “before meeting each other, what 
important is the Internet; after meeting each other, the Internet just becomes a tool for communication. The meeting 
enhanced the online interpersonal relationship.” Furthermore, in the virtual community, people can find and provide 
social support and build their own social networks (Wellman, Salaff, Dimitrova, Garton, Gulia & Haythornthwaite, 
1996). Generally speaking, exchanging information is the basic function in virtual communities . Chatting is 
another. They also build friendships and a sense of belongingness. Sharing information and self- disclosure will 
help them build such interpersonal relationships. 

Examples of virtual communities 

In order to understand how virtual conimu nities work, three virtual communities will be examined. The 
first one is TaNet Elephant-fan Association (TEA), a discussion group on NTUEE Maxwell Bulletin Board System 
(BBS) in Taiwan. It is an online group built in 1995 by people who supported one of Taiwan’s professional baseball 
teams - Elephant Team. The second one is The Whole Earth ‘Lectronic Link (WELL), an online gathering place 
which started in 1985. As The WELL website announces , “the heart of The WELL” is conferences 
( http://www.well.com.conf/conference.html ). The third one is Big Sky Telegraph (BST), which was organized as a 
telegraph conferencing group for teachers’ online learning in rural Montana in 1988. The basic function of these 
three communities is to provide social support and exchange information. According to their function, they are also 
called virtual social communities. As successful virtual communities, there are some similarities and differences 
between these three sites (see Table 1). 

First, an obvious difference is that both The WELL and the BST were built in the 1980s, while the TEA 
was constructed in 1990s. The reason for this is perhaps that the development of the Internet has been faster in the 
U.S. than in Taiwan. The theme of each group is also different. The topic of the TEA is about baseball, while The 
WELL has several discussion topics, such as parenting conference (Rheingold, 2000). Meanwhile, for BST, was a 
group composed of teachers, discussing teaching and learning topics. As to the interface the participants employed, 
BBS is the major interface used by the TEA members; WWW is the base for The WELL, while the telegraph 
technology is for the BST. BBS is the simplest, cheapest, text - based infrastructure and it is a grassroots element of 
the Net subcultures (Rheingold, 2000). As a versatile interface, WWW is not only text -based but also pictures and 
multimedia can be displayed. Telegraph is also a text -based CMC tool, similar to BBS. Fourth, as to the operation 
tools, both the TEA and the BST rely on the keyboard to input the commands, such as posting articles or turning to 
other articles. In contrast. The WELL uses a mouse to “point and click' the commands. Compared to the other two 
groups, in the TEA, people can check other members’ online status to see who is online, and they can send instant 
messages (throw water balls) to each other. Checking online status can create an atmosphere as if “we were 
together”. By sending instant messages, members online can talk to who they want to talk to personally and receive 
instant responses . 

The other difference is the membership fee. It was free for both the TEA and the BST members, but in The 
WELL, it requires a membership fee. As for knowing who you are talking to. The WELL requires its members use 
their real names to communicate because they think using real names can make their conversations and relationships 
real, so does BST. But for the TEA members, they are anonymous with their NetID acting as their personal identity. 

Although they have many differences, there are still some similarities among these sites. Both the TEA and 
The WELL provide member list and require members to disclose their basic personal information. In addition, the 
TEA and The WELL provide nickname function, which can show membeis’ mood, thought or whatever they want 
to share with other members. All three sites have hosts to manage groups. In the TEA, one of the responsibilities of 
the host is to sort daily articles and collect important articles into digests so the members or visitors can review what 
other members discussed previously, and read some important game records. Last, all of them provide opportunity 
for members to see each other in real life. Despite one of the characteristics of the Net is anonymity, along with 
their community development, members of the TEA and The WELL developed face-to-face meetings. It is not 
mandatory. On the contrary, it is voluntary. Members of the TEA and The WELL indicated that meeting each other 
in person is quite important to build their relationships. For the BFT, it is not mentioned that their annual meeting is 
mandatory or voluntary, but the author Uncapher (2000) also pointed out that once the members of the BFT met 
each other, they could have deeper and more meaningful discussions. 

The themes of these three groups are different, but they are based on common interests and set hosts to 
organize group activities. Personal identity is shown on the screen (ex. nickname) and other members’ information 
can be read, too. Some self -disclosure is necessary to let others know who you are and to exchange information. 

As a matter of fact, each group has developed their own patterns to run their group dynamics. 
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Table 1 Comparison between TEA, The WELL andBST 





TEA 


The WELL 


BST 


built year 


1995 


1985 


1988 


theme 


baseball 


parenting 


Teacher 


interface 


BBS 


WWW 


Telegraph 


operation tool 


keyboard 


keyboard, mouse 


Keyboard 


online status check 


yes 


unknown 


Unknown 


instant message 


yes 


unknown 


Unknown 


cost of membership 


free 


member fee 


Free 


personal information 


required and open 


required and open 


Unknown 


member lists 


yes 


yes 


Unknown 


anonymity 


Net ID 


real name 


real name 


nickname 


yes 


yes 


Unknown 


host 


yes 


yes 


Yes 


providing digests 


yes 


unknown 


Unknown 


face to face meeting 


yes 


yes 


Yes 



In face-to-face meetings, people can see other members ’ facial expression or body language. Also, people 
can express their emotions online by using symbols or graphs. For example, in MSN messenger system, emotional 
expressions are provided (see table 2). With these symbols or graphs, people can see others’ expressions more 
concretely. However, emotional expressions differ from interfaces. On the BBS interface, it is convenient to 
conduct commands by keyboards, but people can just use simple symbols, instead of graphs. It takes more time to 
create graphs on the BBS interface. On the contrary, it is easier to do so by either symbols or graphs on the WWW 
interface. WWW also allows voice or real time camera transmission. Therefore, using WWW is more 
advantageous to communicate these expressions. 

Table 2 Graphs and Symbols of emotional expressions 

Graph Symbol Graph 

'J;: Smile :-) or :) Open-mouthed 

$ Surprised :-0 or :o ^ Tongue out 

Online learning community 

Similar to the three examples described above, online learning community, is one kind of virtual 
community. However, it owns specific function of learning and education. It is a virtual entity which combines 
learning and community together (Downes, 1999). There, one can learn not only online courses but also how to 
interact with other participants. In order to build an online learning community, Clark (1998) proposes three 
principles. First, he emphasizes that a learning community is not built, but grown itself. Clark indicates that a 
community will be strong if it is molded by its members to create its own environment. Therefore, members should 
clearly communicate the purpose of the community, and make guidelines and regulations. Once they can realize 
that they are as one part of the community and support each other, this community will be sustained. Second, strong 
leaders are needed. Leaders are not only responsible for managing the community but also have to adopt the role of 
facilitators. The third principle is that personal narrative is encouraged. Clark asserts that personal narrative is “the 
sun that makes communities grow.” Exchanging experiences or opinions can make members feel closer and provide 
identity. In addition, Downes (1999) also points out that for learning communities, creating a sense of commitment 
is important. Once members can build their trust into this community, they will share their learning and personal 
experiences more. As to the attributes of successful learning communities, Downes also makes several suggestions 
from course management, function of the facilitator, and the tasks of students . For course management, he proposes 
that contents and communication should be integrated together and it is allowed to generate contents by some 
members. Also, multiple resource access should be provided. In terms of the function of the facilitator, the 
facilitator should share his enthusiasm with all members and become involved in discussions. Moreover, this 
facilitator should link members and content together, as a moderator between them. For students or members, they 
should build their trust in this community and establish relationships with others, thus increasing the quantity and 
quality of discussions. 



Symbol 
:-D or :d 
:-P or :p 
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On the website “principles of online design” ( http://www.fgcu.edu/onlinedesign) . some guidelines are 
stated from the perspective of instructional design. For promoting an online learning community, it declares that the 
instructor should have social contacts with students in the instruction plan and creates an atmosphere for sharing as 
well as using some tools, such as e-mail or a discussion board to increase interaction. It is also asserted that in 
online learning, students are required to participate in discussions and interact with others. On the other hand, 
Differding (n.d.) in his online article “preparing students to join the online learning community,” focuses on the 
conversation rather than course design. In Differding’s opinion, informal communication is allowed into 
conversation to build one’s identity. He suggests that jump ing into the course content is not immediately necessary. 
Instead, some space for informal conversation is needed to “warm up” the atmosphere. In addition, students are 
required to introduce themselves and share their interests to others. Although the purpose of the learning 
community is “learning”, Differding suggests that social interaction is strongly encouraged and the instructor should 
design group projects to provide students with some opportunities to engage in one task in order to increase their 
peer interaction and common experiences . 

In other aspects, some scholars assert that face-to-face meetings are important to online learning 
communities (Conrad, 2002; Palloff & Pratt, 2001; Edstrom, 2002). They stress that meetings can motivate students 
and strengthen their social networks. As to the impact of communication format on online social presence, Tu 
(2002) indicates that e-mail is the highest level of social presence, followed by real-time discussion and the third one 
is bulletin board in text -based CMC. 

In sum, to facilitate social interaction in an online learning community, incentives are required to attract 
students to participate, which include the designed activities, face-to-face meetings, and the provision of appropriate 
online communication tools. Once all members are encouraged and participate in the community activities 
enthusiastically, it is possible to build an online learning community. 

From virtual social community to learning community 

As Delahoussaye remarks (2001, derived from Differding, n.d.), because of the separated space, online 
education is “an isolating and lonely experience.” In traditional classrooms, body languages and the interaction 
atmosphere are important elements in facilitating learning. Therefore, how to generate an active, interactive online 
environment is one of many challenges for online learning. As a participant of the TEA, the author tries to propose 
the way to build an online learning community from the view of social interaction. The following analysis is based 
on her online experiences, and other research results . 

According to the group development and roles of both teachers and students, four factors are considered to 
build an online learning community. They are: beginning, activities, communication form and environment. 

Beginning 

Creation of the community is the most important stage. If newcomers can feel comfortable, they have a 
willingness to share their ideas or experiences; thus, his learning community is formed in the right way. In this 
initial stage, the following events will motivate group dynamics. The teacher should act as a facilitator or talk with 
students as a peer. Meanwhile, students will be required to share their personal information or experiences. 

(1) Posting personal information 

For the teacher, before beginning the online course, he or she should decide what personal information will 
be posted, such as gender, and e-mail address, and then asks everyone to post their required information. It 
is mandatory rather than voluntary. The teacher also has to post his /her information, thus encouraging 
students to do so. 

(2) Greetings and informal talk 

The teacher will greet students online first and talk about such us as weather, their interests or other informal 
topics, non-course issues first. Informal talk is one tool to enable people to have a sense of “we”, which 
makes people feel warm and therefore it increases self-disclosure during the conversation. Even one word is 
allowed in these online chats. These informal talks are also permitted when community members start to 
discuss the course. 

Activities 

There are several activities that enable interaction and familiarity with others. 

(1) Group identity 
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In the TEA, there are some ways to form group identity. One is playing games on providing nicknames. In 
one period of time , the TEA members edited their nickname s into the same format, such as constellation + 
name + personality description. When they went to the main member list of the NTUEE BBS, they found 
easily who was a member of the TEA easily and knew other members ’ information by reading the 
nicknames. Another identity creation method in the TEA is to create and produce uniforms which allows 
people to recognize other TEA members more easily when walking on the street. For learning communities, 
the host can design such similar games or rituals to bring people together. 

(2) Making rules 

Another element in creating a sense of group is making rules. The teacher could propose some rules and post 
them online. Then discuss these rules with students. Let all members in this community decide on which 
rules to accept. Making rules can provide members with specific references when there is any question about 
group dynamics. With these adopted rules in place, people will know how to respect others and behave 
properly online. 

(3) Synchronous chatting 

In the TEA, members like to chat at the same time on the discussion board. It is a relay board where people 
post or reply the previous articles . It is a ritual to create the atmosphere of “we were here and togethef ' and 
members have an opportunity to be involved in this community. In other words, it is a sense of participation. 
In the learning community, the host can select a specific time for group discussions, including both formal and 
informal talks . 

(4) Picture showing 

It seems normal that human beings have the tendency to associate the face of someone with the person they 
have talked to. In the TEA, the BBS interface does not allow people to show their pictures. However, on the 
WWW, it is easy to show pictures . Picture posting is an incentive, but it is not recommended to show pictures 
at the beginning. Instead, it is better to do it after some discussions. Teachers can show their pictures first. 
Then at the middle or early middle of the class, ask students to show their pictures as well so that other 
members can know who they are talking with. 

(5) Video conferencing 

In addition to pictures, video conferencing is another way to make virtual life “real” . However, this technique 
needs more supporting infrastructures . 

(6) Face-to-face meetings 

Face-to-face meetings are the most distinct feature for these three virtual social communities. Members in 
these three communities enthusiastically approved of this function. This event really helps in building the 
sense of community. If the members of the online courses are located regionally, the teacher or the host 
should arrange the opportunity to meet together. 

(7) Group projects 

Conducting group projects is a more academic way to gather people together. The teacher should propose 
some collaborative projects in order to allow students to work together. 

Communication form 

In a learning community, in addition to the discussion board, e-mail and listservs helps people to transmit 
messages . If it is possible, the teacher or the host can provide an instant message function because people like to 
receive immediate responses rather than delayed ones. Instant communication makes people more involved in the 
conversation. If an instant message function is not affordable by the teacher, MSN messenger or Yahoo messenger 
systems are alternative choices. 

Environment 

In addition to activity design and providing communication tools, a well-designed environment also 
contributes to participants’ interaction. 

(1) Host 
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The host is the soul of the community, like the president to the country. The Host is responsible for managing 
the group dynamics, arranging the posting articles and digests. Moreover, he or she would design some 
activities to inspire the whole community. Usually, the teacher is supposed to be the host. However, students 
can be the host too. They can regularly rotate the host position. Being the host can make students have a 
sense of responsibility to the community. Also, by serving as a host, they have more opportunities to 
participate in community affairs . 

(2) Online status check 

Being online is an isolated behavior because only you interact with the computer. However, if you know who 
are also online with you, you will not feel alone anymore. Thus, providing online status checking function 
facilitates the sense of “we”. Furthermore, in the online status check function, people can choose who they 
like to have an instant conversation with. 

(3) Providing the nickname function 

As mentioned before, nicknames are one of the formats to enhance group dynamics. Moreover, by using 
nicknames, people can express their mood or thinking without posting articles. Another function of the 
nicknames is that when people read articles, they can know some personal characteristics of the author via 
nickname description. 

(4) Providing emotional graphs or symbols 

Because people can not see each other online, emotional symbols and graphs help people expressing their 
feelings. Sometimes, after long communication periods, people can even create their specific emotional 
expressions which only belong to their community. In a learning community, if emotional graphs or symbols 
are provided, it is believed that participants’ interaction can be facilitated. 

Conclusion 

Although an online learning community is a kind of virtual social community, it still has some distinct 
features. First, it may be a problem to build an interactive environment due to the limitation of an academic period, 
say one quarter or semester. The social interaction here is more condensed than in the TEA or The WELL. 

Therefore, it is questioned that whether the whole group developing process can be built during an academic period. 
Second, in a learning community, people have to accomplish assignments and be graded. These may be the barriers 
of their discussion and idea sharing. Third, how to lead social dialogues into academic dialogues may be a challenge 
for teachers. Even so, it still has great possibility to build an online learning community. The major reason is that 
users today are more familiar with online environments than ten years ago. Before they (both teachers and students) 
go to online courses, most of them have had online experiences for several years. With their digital literacy, they 
will get into situation more easily, thus providing more possibility to build an online learning community. 
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Introduction 

The advent of Video-On-Demand (VOD) brought promises of rich video libraries, robust interactive 
lessons, and powerful tools for the curriculum developer. VOD has brought the excitement of a new media form, but 
with this excitement comes the confusion that accompanies any new field. This paper will discuss the nature of 
Videoon-Demand, and explore VOD’s impact on education and curriculum development. We hope to clarify and 
accurately describe VOD and its marketplace to the educator. 

Next we will show how VOD represents a robust and truly new medium for distance and interactive 
learning by starting with a review of Gagne’s nine steps of instruction and then transposing theses methodologies 
onto the VOD space. Further, we will highlight elements of instructional design unique to VOD curriculum 
development and point out the power these elements pose. 

We will continue by describing a generic VOD lesson plan, and then show the process by which a VOD 
lesson is programmed. Finally, we will walk through a completed VOD lesson finishing with a description ofthe 
future of VOD and its place in the arena of education. 

Journey from Wasteland to Oasis 

Americans devote more hours to watching television than to any other media - averaging 4 hours per 
person per day. TV is usually the center of the home and is on in the average house at least 7 hours every day 
(Nielsen Media Research, 2000). Forty-Five percent of all parents report using television to occupy their child 
(Kaiser Family Foundation, 2003). In the past 15 years Cable has transformed televisbn from the nightmare of 
“The Vast Wasteland”, to a 500 channel universe where viewers are more informed about the world while being 
entertained and educated on a variety of subjects from history to business, and from political science to forensic 
science. 

Without any formal effort Cable has become the primary source of learning for tens of millions of 
Americans and hundreds of millions of people around the globe. Nearly 70 Million homes in the United States have 
Cable TV, and over 95% of the digital cable households will have access to VOD programming by 2008 (Paul 
Kagan & Associates. 2004). Information and education channels are the world’s most watched cable channels : 
Discovery Channel is ranked #1 in cable networks with a subscriber rate of over 88 million, and nine of the top 
twenty cable networks are educational or informational (Kagan Research LLC, 2004). 

Unlike broadcast or satellite television. Cable TV has “two-way” functionality built-in, and thus can 
become a truly personalized medium. The Telecommunications Act of 1996 allowed cable operators to invest vast 
sums of money into cable upgrades and technology systems, thereby expanding the frontiers of cable technology and 
allowing for intriguing new services. Since 1996, the cable industry has invested more than $85 billion to rebuild 
and upgrade its facilities, including $10.6 billion in 2003 alone (Witness testimony at the US government 
Subcommittee on Telecommunications and the Internet; Kevin Leddy, Sr. VP, Strategy and Development Time 
Warner Cable, May 19, 2004). These new cable technologies, particularly VOD, are giving instructors new tools for 
more effective instruction and new ways of reaching those who want to learn. 

What is VOD? 

Defining “Video-on-Demand” is often in the eye of the beholder. Experts and laymen often refer to VOD 
in a myriad of terms from “movies any time you want” to “streaming media”. Sometimes VOD is merely defined 
as “the ability to pause and rewind TV shows without the need for VCR.” These definitions are only partially 
accurate, and more often are misleading. For the purposes of this paper, we will define VOD by three elements: 
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1. Digital media. 

Video must be prepared (encoded) into a digital format - e.g. MPEG-1, 2, and Windows Media. Digital 
video can be manipulated in such a way as to provide meaning and context. Since digital video is a medium 
prepared in discrete chunks, as opposed to analog media, digital video is easily recognized and processed by 
software. 

2. Delivered over Broadband (Cable) Television 

Cable - VOD is delivered without any delay and thus is not “streaming media”. Streaming media typically 
has a series of “buffering” delays in delivery, usually there to overcome memory limitations in personal computers. 

Second, VOD is viewed on a television set, and delivered via “set-top box”. The television viewing 
experience is essential not only for video quality, but because of the mindset and behavior of the viewer/learner 
when sitting in front of a television versus a personal computer or any other learning experience, including with a 
live instructor. 

Third, all media (video, audio, and graphics) originates remotely but are delivered in such a way the viewer 
is unaware as of their location of origin. They see it immediately or “on demand” (hence the term Video-on- 
Demand). 

3. Incorporates software intelligence 

Incoming media must have additional software added in the TV “set-top box” that helps the learner interact 
with the media . As a result the media changes and adapts depending on the viewer’s response or behavior. 

Misconceptions and Differences 

When discussing VOD it should be noted that there are a number of misconceptions preventing many 
designers from accepting VOD as a valid educational tool. 

1. Education on cable via VOD is not the same as “distance learning”. 

Receiving VOD does not necessarily require that there be “someone awake at the other end” to receive the 
information. Video-on-Demand can either be a passive experience, a background experience or a shared group 
experience with “no one at the wheel”. 

2. A keyboard is not essential for the initial educational experience. 

Vast interactivity can be achieved using a conventional remote control, and frequently encourages more 
interaction due to the fact that the remote control is a simple device, requiring minimal dexterity. 

3. VOD programs are not presented the same for all viewers. 

Software intelligence that chooses and manipulates video can be combined with the cable operator’s ability 
to store and recall user data, to create truly customizable programs tailored to the skill level and interest of the 
viewer. 



New Technologies Make VOD A Powerful Force for Education 

While VOD learning has facets in common with other media/multimedia learning systems such as 
DVD/CD-R, and Personal Computers, the following features, when combined with software intelligence, transform 
Cable -TV VOD into a unique and powerful new medium for learning and for teaching: 

1. VOD has true interactivity 

The ability for the learner to use their remote control in order to respond, real time, to questions posed by 
“the media” (an instructor depicted in the video, “talking” to the student, or simply a graphic “True/False” menu). 

2. VOD has Branching 

Branching is the ability to “jump” to different segments of the video - driven by viewer control, or behind 
the scenes, without the viewer being aware of the change in flow. 

3. VOD has Intelligence 

Based on viewer/learner responses, and based on the curriculum developer’s (instructor’s) design, the 
software can change what kind of video, graphics, audio or other information the viewer sees. In other words - the 
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software can learn about the learner - and present educational media that can guide the learner, become 
personalized, adapt and challenge the learner to become a highly effective educational experience. 

4. VOD Capitalizes upon TV Viewing Behavior 

People behave differently when in front of a television, and their expectations are different than almost any 
other medium. TV is a “persistent” and active medium: When TV is on it is always delivering content and 
changing every minute (as compared to a static computer screen). TV viewing behavior is much more dynamic 
than most realize. First, viewers are used to dividing their attention between different channels, often keeping track 
of several stories simultaneously. Second, though TV was originally a one-way, lean-back, storytelling medium it is 
now anything but a passive experience. Viewers anticipate shows and events, modify their behavior and lifestyle 
around scheduled airtimes of their favorite shows, patiently await the next episode, willing to be interrupted, and 
will dutifully return to the TV after the commercial break. They become passionate about the content and 
characters, and even more passionate about which channels and information they watch and trust. 

These behaviors, combined with VOD and software intelligence, create a unique new environment for 
learning and teaching. This VOD learning environment enables educators to: 

• Create lessons available to broad audiences but automatically personalized and customized to each learner. 

• Combine many different kinds of media video, audio, graphics, along with other modalities (kinetics - by using 
the remote control) to impact the learner in a highly personal and effective manner. 

• Combine pre -defined course media, with personalized responses and pace. 

• Combine pre -defined course media, with timely, real-time information (e.g.: VOD- Instructor to Student: “Use 
your remote contralto identify 5 members of the President’s cabinet on any of the news channels tonight”). 

• Reinforce in a variety of new and effective ways (e.g. for someone learning French, the lesson can, during a 
movie occasionally pop-up the French translation for a familiar line in the movie). 

• Allow the student to take control and guide the educational journey on a path that suits her or his interest. 
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A framework for VOD as a teaching tool: 

The work of Robert Gagne is particularly relevant to VOD instruction. In 1970, Gagne outlined 9 steps in 
the instructional process that applies well in today’s new media, interactive age. Gagne stated that there are nine 
events that should occur to stimulate effective learning (Gagne, Robert M, 1970, The Conditions of Learning 2nd 
ed., New York; Holt, Rinehart and Winston.). 




1. Gain attention: Present a problem or a new situation. Use an "interest device" that grabs the learner's attention. 

2. Inform learner of Objective: This allows the learner to organize her thoughts and around what she is about to 
see, hear, and/or do. 

3. Stimulate recall of prior knowledge. This allows the learner to build on her previous knowledge or skills. 

4. Present the material. Impart the information in manageable portions in order to avoid memory overload. Blend the 
information to aid in information recall. 

5. Provide guidance for learning. This is not the presentation of content, but the instructions on how to learn. Usually 
this is more straight-forward than the actual subject matter or content. 

6. Elicit performance. Practice by letting the learner do something with the newly acquired behavior, skills, or 
knowledge. 

7. Provide feedback. Show correctness of the learner's response, analyze learner's behavior. This can be a test, quiz, 
or verbal comments. The feedback needs to be specific, not, "you are doing a good job." Tell her "why" she is doing 
a good job or provide specific guidance. 

8. Assess performance. Test to determine if the lesson has been learned. Can also give general progress information 

9. Enhance retention and transfer. Inform the learner about similar problem situations, provide additional practice, 
put the learner in a transfer situation, and review the lesson. 
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Instructional concepts specific to VOD 



How and where do VOD techniques fit into 
instructional methodolgies? On an abstracted 
level, VOD instruction is straight-forward, and 
consists of 3 elements: Impart-Interact-Adjust. 

1. Impart: Media is delivered to the learner in 
context (at the right time and place). 

2. Interact: Learner input is solicited via 
questions, menus and other methods. 

3. Adjust: Software intelligence decides how 
best to proceed with the lesson based on 
what it observes about the learner and based 
on the way the instructor wants to proceed. 



Fig. 2 Basic VOD Instructional Unit 



Deliver in context 




based on input & decisions other methods 



It is important to note that VOD, along with related control software, is essentially independent of the subject matter 
or content of the media. Instructional video is displayed to the viewer and manipulated by VOD, but for the 
purposes of this discussion VOD does not create the video or graphical instructional materials. 



Guided vs Self-Paced 



Depending on the “script”, the learner can be guided through the lesson in a manner pre -determined by the 
instructor, or the learner can take control and embark on a completely “self paced” lesson. Between these two 
extremes, there is a powerful mode where the pace, level and subject matter of the guided instruction changes 
depending on what the instructor learns from the student. Based on feedback from the learner (number of incorrect 
answers, slow responses, frequent repetition of information), the order, pace, and content of the lesson can change in 
order to better teach the student. 



Teach and Assess 

The simple “impart-interact-adjust” model can be used as part of a broader abstraction to elicit two major 
instructional mechanisms: Teach and Assess. 

Fig. 3 Guided Vs. Self-Paced Instruction 
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Media (lessons) are in a specific order 
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Guided 




"Self Paced 



Learner guides the pace and order of the 
instructional media 



Planned Tests, 
Quizzes, side- 
bars 




Pop Quizzes, 
& Surprise 
Reviews 
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Table 1 - Teach & Assess 



TEACH 


ASSESS 


Imparting of pure instructional media (information, 
lessons, examples etc). 


Determining the learner’s progress through various 
challenges (tests) and other methods of determining 
how well they met the learning objectives. 


Guided 


Self-Paced 


Guided 


Self-Paced 


Video, Audio and Graphics 
are delivered in an order 
and pace predetermined by 
the instructor 


Learner controls the 
pace and order of the 
information 


Planned tests, 
quizzes and other 
challenges are 
dictated by the 
instructor 


Learner determines when 
he/she is ready to test, can 
self-test and review. 


“Adjusted” 

Based on interaction and feedback from the 
learner, the pace, flow, content and context 
of the information changes and adjusts to 
the level of the learner. 


“Adjusted” 

Based on interaction and feedback from 
the learner, questions or challenges are 
changed dynamically in a way that 
focuses on the learner’s objectives, or 
weak points. 



Anatomy of a VOD Lesson 

Each basic “impart-interact-adjust” VOD element can be used and re-used to create meaningful instructional 
modules that can be combined - either during curriculum development-time, or real-time - to create a robust 
framework for instruction. 



Fig. 4 Anatomy of A VOD Lesson 
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Figure 4 and accompanying Table 2 detail various VOD learning modules (each constructed of a portion of the 
simple “impart-interact-adjust” module). These modules parallel the instructional elements outlined by Gagne’s 
Nine Steps. Together they form an almost comprehensive framework for instructing and learning using VOD. 
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Table 2 VOD Techniques vs. Gagne’s Nine Steps of Instruction 



VOD Module 


VOD Behavior 


Gagne Step 


Preview, Arousal, 
Relevance 


Shows segments of the entire lesson up-front to inform and 
heighten interest. 


1 Gain attention 


Objectives, Goal 
Setting 


Spells out the learning objectives via on-screen graphics - allows 
for direct and immediate investigation of the subjects. 


2. Inform learner of 
objectives 


Review 


Automatically present summary of prior segments. Remembers the 
learner’s level and reviews to the point of the last learning session. 


3. Stimulate recall of 
prior info 


Instructional 


Straight-forward playing of instructional video, with timed, in- 
context, graphics and text information. 


4. Present the info 


Sidebars, Examples 


In -context guidance (video, text), based on detection of learner 
level of proficiency. 


5. Provide guidance 
for learning 


Quiz, Challenges, 
Exercises 


Pop-quizzes, questions and other on-screen mechanisms that 
challenge the learner to respond with answers. 


6. Elicit performance 


Feedback 


Frequent on-screen information as to how quickly the learner is 
progressing, or how accurate learning is thus far (based on 
quizzes). 


7. Provide feedback 


Test, Assessment 


Presents the learner with a series of questions in a variety of media 
and formats, in order to gauge how much of the lesson was learned. 
Assesses level based on score, response time and other metrics. 


8. Assess performance 


Reinforcement 


Media imparted with simple or sophisticated rules to provide 
variety, randomness, and context - via demonstrations, 
motivational segments and “linked” segments. 


9. Enhance retention 
& transfer 



VOD Implementation of Learner-Centered Instruction 

Next, we move from concept to design, then to implementation: How do we design lessons using VOD 
modules, and how are these modules actually implemented? First, we need a system where we can “feed in” media, 
and process instruction-oriented commands that transform the media into a lesson. This system must deliver the 
media seamlessly to the learner, accept input (interaction, responses) and then change the mix and flow of the media 
accordingly. The curriculum developer must have a simple framework for specifying what information will be 
imparted to the learner, and how the lesson will change based on the learner’s interaction. 



Fig. 5 Real-Time Delivery of VOD Instructional Media 
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Designer specifies how 
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Software “engine” in Cable 
Set-top box - processes 
instructions, manipulates 
media and detects input from 
the viewer/learner 



Feedback, interaction and 
responses direct the software to 
make decisions, and to change the 
flow of the lesson, adjust the pace 
and content of the instructional 
media shown to the learner 



Software technology by Chaos Media Networks is designed to allow the curriculum developer to specify the VOD 
learning modules, the related media and decision-making rules. These rules are executed “real-time” and the 
learning media experience is created as per the designer’s intentions while the learner/viewer is watching TV. 
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Preparation Steps for VOD Curriculum Design 



Educational VOD still relies on solid planning and curriculum design practices. Planning and preparation are as 
powerful as the new delivery mechanism enabled by VOD. Below are the planning steps: 

1. Identify the instructional content. Gather video, audio or graphics to be used for the raw instructional assets. 

2. Determine the flow of media. Decide the general order in which media assets should appear in the lesson. 

3. Determine points of participation, interaction or control: Where learners can deviate from the planned lesson flow. 

4. Determine the challenges: Quizzes, tests etc. 

5. Determine proficiency / difficulty levels: Decide which media is appropriate for different competency levels. 

6. Decide assessment levels: i.e. decide how to assess the level of learning (scores, speed, etc.) 

7. Identify and provide various areas of feedback, reinforcement, motivation, demonstration - in the form of in- 
context sidebars, games, challenges and other techniques. 



Authoring for Real-Time Use 

Intelligence provided by the Chaos VOD software engine allows the curriculum developer to list out the VOD 
teaching modules outlined earlier (Figure 4, Table 2). These commands are listed in tabular form, in a spreadsheet, 
purely for the convenience of the designer. 

Table 3 - VOD Scripting Commands 



Scripting Module 
Command 


What happens 


Instruct 


Plays media* in a specific predetermined order without interruption. 


Guided 


Same as “instruct” but allows the learner to control the speed of the instruction and 
jump around to specific segments via a menu designed to guide them back to the 
proper segments. 


Objecti ves 


A graphical display of the goals of the particular section, which can be adjusted based 
on the progress of the learner. 


Explore 


Allows the learner to take control of a particular section and view the media without 
interruption or guidance from the system. 


Quiz, Pop-Quiz 


Within a particular guided/instruct segment, a random, in -context quiz is made to 
appear - challenging and assessing the learner’s progress. Quizzes take on a variety of 
modes from simple multiple choice, to more complex game -like challenges. 


Test 


Specifies a series of questions at the end of a section in order assess a score. 


Adjust 


Allows any subsequent media and assessments to change based on responses from the 
learner (score, speed, completion of sections). 


Other 


Several other commands: Reinforce, motivate, remind, review, demo, sidebar, think 
(“think about it”), assignment are all different methods of imparting instructional 
media in -context, and in a variety of different visual delivery modes in order to make 
learning more effective. 


Status, Progress 


Allows the designer to query and/or display the learner’s status (completed segments, 
correct answers, speed, duration). 



* Media = Any combination of video, audio, graphics or text. 
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A Sample Instructional VOD Design and Session 



For purposes of demonstration, we used a video from the Agency for Instructional Technology: “Inside Our Body: 
From Ear to Hear” (AIT, 1998). In this sample demonstration we use the following script . . . 



1. PREVIEW the lesson, and start from where the learner left -off last. 

2. OBJECTIVES - List objectives for the current segment. 

3. GUIDED - Present the lesson, but allow the learner to explore at their own pace. 

4. POP-QUIZ - at a specific time display a surprise-question to assess the learner’s understanding. 

5. SIDEBAR, THINK, DEMONSTRATE - show extra information and give opportunities for the learner to explore 
deeper or in more unusual directions before resuming the lesson. 

6. GUIDED - Continue lesson, with more quizzes and sidebars. 

7. ASSESS - determine progress by comparing pace and scores on quizzes. 

8. ADJUST - Revisit specific sections in more depth, or accelerate pace based on assessments. 

9. TEST - Formally assess the learner’s understanding with a series of questions. 

10. FEEDBACK - Display and explain results, review and give recommendations for future learning sessions. 



Fig. 6 Ear to Hear VOD Demo Flow 
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Summary of VOD Demo Experience 

Figure 6 illustrates (in an abridged format) how this session flows. Note that interactive VOD sessions are 
not “top-down” nor typically have a beginning, middle, and end. While instructor (designer) has loaded the 
knowledge and decided some of the flow and rules, it is the learner that is in charge, with the program/lesson 
reacting and adapting. If the learner is going slowly, the lesson spends more time in problem areas. If the learner 
answers questions correctly, the lesson picks up pace. If the learner simply wants to indulge their curiosity and 
explore - they can spend time on that too. And if the learner has a specific objective, perhaps only this one time, 
they can specifically direct the lesson to focus on this objective. 

The technology is designed so that the scripting commands are abstract and modular, so the designer can 
focus on the flow and the media, as opposed to technology. The scripting technology/framework is designed so that 
the modules “reuse” themselves, and also displays media in a variety of random formats in order to always give the 
learner a fresh and engaging experience. 

Conclusions / Recommendations 

Video-on-Demand over Cable is a new and powerful medium for teaching and learning. It combines non- 
linear video and software intelligence with the unique viewing behaviors of television. Interactive Video-on- 
Demand spawns many new design concepts that dovetail very easily with common instructional techniques. Using 
special software that enables both the design and delivery of interactive VOD instruction, curriculum developers can 
now produce lessons and courseware that can reach broad audiences, while having each lesson be a highly 
personalized learning experience for the student. 

The basic functional modules of VOD are simple: Impart -Interact-Adjust. When combined with a 
repository of raw instruction media, the combinations are endless. When designing for VOD instruction, the rules 
and flow of the lesson are as important as the actual instructional media. Video, audio, and graphic assets can be re- 
used and re-purposed to create seemingly original instructional products. These new instructional products adapt 
and personalize themselves real-time so they appear differently to each learner. 

While curriculum developers might not need to know deep technology in order to script new VOD learning 
products they will need to think differently. Certainly the conceptual design process needs to focus on the ‘non- 
linear’, but also on using existing assets, and targeting many different types and levels of viewers, and adapting. It 
means designing in a ‘behavior’ and intelligence into the learning product that will know how to respond and adapt 
to different learners and different situations. 

Within the next 3 years Video-on-Demand will be available in almost every cable TV household. 
Curriculum developers now have the opportunity, means and methods to tap into a $150 Billion home educational 
media market so far not possible over television. Also within the next 3 years, Cable television systems will allow 
additional functionality such as voice recognition, social networking, and two-way video - giving curriculum 
developers almost unlimited possibilities to reach and teach students. 

After nearly 10 years and over $80 billion dollars in digital cable infrastructure development, the ‘golden- 
spike’ has finally been driven. The opportunity and possibilities have increased by several orders of magnitude 
almost overnight. Video-on-Demand over cable television is about to give curriculum development a new 
renaissance, and the time to start learning the VOD-specific concepts and techniques is right now. 
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Technology Grants and Rural Schools: The Power to Transform 
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The requirements of No Child Left Behind Act of 2001 (NCLB) has presented challenges for schools and 
districts across the United States such as a new need to focus on test scores and student achievement. While all 
states, districts, and schools face challenges that require them to adjust the structure and delivery of instruction in 
their schools, the small population and geographic isolation of rural schools can make change even more challenging 
(Reeves, 2003). Some have suggested that one way some rural schools may be able to overcome these challenges is 
through an increase in the level of technology integration in their school (Collins & Dewees, 2001). Schools struggle 
not only to implement and integrate technology into their curriculum, but also struggle to find the funds that they 
can allocate to the purchase and maintenance of technologically -enhanced instructional strategies. Fortunately, the 
high cost of technology and the potential educational impact of technological resources have led to the awarding of 
federal and state grants to facilitate the implementation of educational technology in schools (Flerr & Brooks, 2003). 
This study examines one school’s attempt to use grant money to purchase and integrate specific instructional 
technology into their school in order to increase student achievement. 

Rural Schools 

Schools in rural areas or small towns make up nearly 42 percent of all schools in the United States and 
represent 30 percent of students in the country (U.S. Department of Education, 2002). A rural school is defined as a 
school in a community whose population is less than 25,000 people (Mathis, 2003). These schools face many 
challenges due to their unique characteristics including: geographic isolation, declining enrollment, small 
population, limited funding, and lack of access to services (Reeves, 2003). Further compounding the challenge is the 
frequent use of funding formulas that allocate funds to districts on a per-pupil basis. These formulas are often used 
by federal and state agencies to distribute money to schools and puts rural schools at a disadvantage as they attempt 
to supplement their budgets (Hadderman, 1999). The availability of funding for rural schools often impacts their 
ability to access programs, services, and training opportunities, and plays a role in their inability to build capacity to 
comply with the standards set forth in the NCLB Act (Reeves, 2003). 

Technology and Teacher Attitude 

Teacher attitudes toward technology influence the level of technology integration in schools. According to 
the National Center for Educational Statistics (NCES), less than 20% of teachers reported feeling very well prepared 
to use technology in their classroom instruction (USDE, 2002). Training teachers to integrate technology is another 
way rural schools can invest educational technology. Once rural schools have successfully recruited “highly 
qualified” teachers and provided them with technology, schools must provide ongoing training in technology as well 
as administrative support in order to facilitate successful implementation of technology-facilitated instruction 
(Wang, Johnson, & Pisapia, 1994). Heath et al. suggest two factors that influence teacher attitude change toward 
technology integration are (1) having a willingness to change, and (2) the control structure of the school 
environment. Allowing teachers to see the potential benefits of technology for themselves and their students may 
help facilitate an attitude of willingness to change. Additionally, maintaining a power structure in the school that 
allows teachers the freedom to move from one stage of technology integration to the next in a supportive and non- 
dictatorial manner allows teachers to feel empowered to introduce technology into their instruction. Heath et al. 
(2000) also found that professional development and training in technology enabled many teachers to integrate 
technology effectively. Providing opportunities for exposure and development of positive teacher attitudes toward 
technology, is the beginning of the change process as schools try to move toward technologically integrated 
instruction. 

The nature of rural schooling, teacher attitudes toward technology, and the utility of technology to impact 
teachers’ instruction, are all factors that must be considered as schools look to provide an education for students that 
optimizes learning opportunities and provides cost-effective instruction. The potential impact of technology to 
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influence student achievement and school performance in this “age of accountability” for schools, raises questions 
about the actual outcomes and processes related to a grant-supported infusion of technology. 

Research Questions 

In order to better understand how technology could change attitudes and practices at a rural middle school, 
we focused on two research questions: (1) How does a grant-related influx of technology in a rural school affect 
teacher and administrator attitudes toward technology use/integration? (2) How does a grant-related influx of 
technology in a rural school affect the integration of technology in teacher planning and delivery of instruction? 

Setting 

Context 

Ed Tech Competitive Grant Program. This grant program is funded by the Federal Government through 
NCLB but administered through state departments of education. School corporations are eligible for grants of up to 
$300,000. A grant application is required to have clear goals and objectives related to the school improvement focus 
of the district. In this particular state, 19 schools were funded in 2003-2004 academic year. Schools had to meet one 
or more of the following criteria to qualify for the Ed Tech Competitive Program grant: either it is one of the school 
corporations (a) among the highest poverty districts in the state, or it is a district (b) identified as in need of 
improvement according to the state based on state standardized test scores (Office of Learning Resources, 2004). 
This study focuses on only one school district that received the grant. While it may have similarities to other schools 
that qualified for the funding, each district is its own unique case. 

Community School District 

Community School District is comprised of four elementary schools, one middle school and one high 
school and has a total district enrollment of approximately 3,065 in a county with a population of 17,281 . This 
qualifies it as a rural school as defined by Mathis (2003). The district has 3 elementary schools, one middle school 
and one high school. The focus of this case study is the middle school environment, because this is where the 
technology grant was targeted. 

Implementation 

The stated goals of the district in the approved grant application were: 

o The percentage of 7th and 8 th graders passing the 2004 ISTEP+ English/Language Arts test will 
increase by 10 points over the 2003 percentage. 

o Teachers will employ three new teaching strategies during the 2003-2004 school year. 

o Average student performance on the reading portion of the Standards-Based Adaptive Measure Test 
will increase by at least one grade-level equivalent. 

(Technology Coordinator, 2003) 

The school’s efforts to meet these goals included the purchase of fifteen laptop computers for teacher-use 
in planning the integration of technology into their curriculum. Another thirty laptops were included for student 
classroom use. Teachers were chosen for participation based on their ability to attend a summer workshop, and 
represented both core (language arts, math, science) and other content areas. 

Three software programs were used to support district goals. Inspiration™ was used to allow the graphic 
organization of student-generated ideas for writing assignments (Inspiration Software, 2004). Additionally, once a 
quarter, teachers used Socratic Seminar™ with the expectation that student writing skills would improve across the 
curriculum as measured by a rubric -scored periodic writing prompt (Technology Coordinator, 2003). And finally, 
an Integrated Learning System, PLATO™, was used to promote reading across the curriculum. Each program was 
chosen based on quantitative research studies, meeting the needs of the grant, and the NCLB Act for scientifically 
based research (Brush, 2002; The Institute for the Advancement of Research in Education (IARE) at AEL, 2003). 
Professional development, including a summer workshop and monthly professional development workshops, 
assisted the district in reaching their goals. The training included workshops regarding the use of the laptops for 
teacher planning, the use of PLATO™, Socratic Seminar, and Inspiration™ software with students, and developing 
technology rich lessons. 

Participants 
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The participants in the study were a convenience sample of teachers from the faculty of a rural Midwestern middle 
school. Interviews began with administrators who were gatekeepers to other participants. A snowball sampling 
method was used by which the initial participants recommended other teachers to interview, fifteen interviews were 
conducted using this method. 

Each participant was a volunteer who had in some manner interacted with the grant technology and 
training, or was a teacher in the target building for the grant. These volunteers were derived from three groups: ( 1) 
teachers participating in the grant training who received teaching resources related to the grant, (2) teachers who did 
not participate in the grant but who taught in the same building, and (3) administrators who helped write or 
implement the grant, those that supported teachers involved in the grant, or building administrators that were 
directly impacted by the presence of the grant. Administrators included the local building principal, assistant 
superintendent, district technology coordinator, a building media specialist, and a district technology support 
professional. Teachers were representative of various levels of teaching experience and content areas that included: 
language arts and reading, social studies, science, mathematics, and also special education. 

Methods 

The study was conducted as a multiple, qualitative case study of administrators and faculty working in rural 
school district in order to examine how receiving a technology grant might impact a school community. For the 
purposes of this study, qualitative research is defined as that which “seeks answers to questions that stress how (sic) 
social experience is created and given meaning” (Denzin & Lincoln, 2003, p. 13). Also, for this research study, a 
case study is defined as “a phenomenon of some sort occurring in a bounded context” (Miles & Huberman, 1994, 
P-25). 

Data Collection Procedures 

District and state-generated grant applications, reports, and other relevant documents presented by the 
district to the state agency that administrated the grant, were obtained from the district administrators both in paper 
and electronic form. Each of these documents was reviewed in conjunction with the data obtained using one of two 
primary methods- interview and observation. Individual teacher and administrator interviews were conducted and 
recorded on audiotape. These interviews were transcribed in their entirety. 

Interviews. Interview data was recorded using audiotape in order to capture the responses of teachers and 
administrators, and transcribed in order to record their thoughts, experiences, and self-report interactions related to 
the grant. Interview questions were based on the overarching research questions; yet, the interviews were semi- 
structured to allow follow up questions and further probing. Questions focused on the impact of the technology 
grant on the environment, teaching practices, and attitudes of teachers and administrators that may have been 
impacted by implementation of the grant. 

Classroom observations. Some participants were asked to allow a researcher to observe their classroom 
during a lesson utilizing technology. Notes were taken by the observer, but no video or audio tape was recorded, and 
no students were recorded. The researcher made general observations about the classroom, teacher and student 
behavior, and classroom activities. While technology use and integration was the focus of the observation, the 
overall classroom experience provided insight into the culture of the school. 

Data Analysis 

The collected qualitative data was analyzed using standard coding procedures as suggested by Gall et al 
(1996) and Denzin and Lincoln (2003), in order to identify emerging themes to support research conclusions. These 
procedures included the coding of repeated ideas and the collection of these topics into broader themes. Once these 
themes were identified, they were classified into relevant categories for later interpretation and use in supporting the 
findings of the researchers. Specific methods of analysis for the interviews and observations are described below. 

Interview and observation analysis. Transcription of fifteen taped interviews was conducted by each 
interviewer in the interests of accuracy and completeness. Each transcript was typed verbatim from interview tapes 
and provided for coding and analysis. The number of researchers was limited to two writing and coding the 
transcripts, for the purpose of establishing a relevant coding scheme and generating useful themes for explaining the 
results. This analysis reflects procedures recommended Denzin and Lincoln (2003), Gall et al, (1996), and 
Carspecken (1996). 

Handwritten notes from the observations were typed and were included for consideration as themes were 
drawn from the interviews. These observations served to triangulate the findings. Triangulation, a strategy for 
verifying the internal validity of the documents, was conducted by correlating the observations with participant 
interviews and the examination of received documents (Gall et al., 1996). Themes generated by the two researchers 
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were debriefed by other team members. This collaboration allowed those group members who had conducted the 
original interviews and observations to verify that the themes were accurate and allowed a consensus to emerge 
related to the findings and implications. 



Results and Discussion 

Through our fifteen interviews, trends began to emerge. The most prevalent trends were related to the first 
research question, “How does a grant -related influx of technology in a rural school affect teacher and administrator 
attitudes toward technology use/integration?” In relationship to this first question, while both groups may have 
shared positive attitudes, it became evident that their overall goals differed. Though only having two observations, 
some gains related to the second question, “How does a grant-related influx of technology in a rural school affect 
the integration of technology in teacher planning and delivery of instruction?”, were observed through teachers 
sharing their own experiences. 

How does a grant-related influx of technology in a rural school affect teacher and administrator attitudes 
toward technology use/integration? 

Teachers shared how having the laptops allowed them to use more technology. The portability of the 
laptops helped to increase their comfort level with technology. A common comment reflected how they could now 
use the computer at home, on their kitchen table, in their recliner, or in their family room while watching television. 
Being able to take the computer home gave the teachers a greater sense of ownership and they found themselves 
using the computer for personal tasks. For example, one teacher related that she started to use power point with her 
Sunday School classes as well. Another was using computer applications for her home business. 

Support 

Teachers related that that the time was important to assist them in using technology. As one teacher 
related, the workshop allowed her time to set up the computer into a usable form 

I thought the most useful part was having time to get to use the laptop and start storing things that you 
could use in class during that four-day session. I really used that a lot. Since they were brand new they did 
not have RealPlayer downloaded and we got all of that done. And usually that is the kind of thing that you 
put off because it takes so long to do and you are teaching. So that was wonderful I thought. 

The social studies teacher had used the extra time given to him to use the Inspiration™ software himself, 
and found that the chance to experiment was vital in deciding if he would use it or not. “That is one thing, if the 
teacher doesn’t feel comfortable using it; they are not going to use it.” 

In addition to benefits stemming from district support in the form of increased time, teachers related that 
their collaboration efforts have increased. Teachers in the laptop program work together to solve similar problems 
and ask each other for help. A teacher who is a novice in computer technology stated “It gets me around the building 
a little bit more so I can talk to my colleagues that I haven’t seen for awhile.” One teacher related how she worked 
with less experienced teachers to get their gradebook software working properly. Teachers related that they felt they 
could ask each other for help when they needed it or the technology support people were unable to provide an 
answer. One teacher noted an increase in communication. 

Email has probably increased 500% in the building, where teachers will communicate. I think 
communication is better on some level. So yeah, I think that improved that. And the collaboration, Mary is 
on the other team, she’ll say that is a good idea, and maybe she will want to try that with her team. 

School -wide support. In addition, to the summer workshop, there was just-in-time support available 
throughout the school year. In Community Middle School, there is a media specialist who is very accessible to 
teachers. When interviewed, she recognized the importance of her role in the success of the grant, “And I know that, 
unless you have someone there who can fix problems, teachers will quickly become frustrated and won’t use it any 
more, they give up. They just won’t use it.” She was a problem solver and helped teachers when they struggled with 
the technology. The portability of the laptops made this even easier. “So a lot of the time, if I have a quick 
technology question, I will just pick up my laptop and go sit and ‘What do you do here?’ ‘How do you get this?” 

The media specialist worked closely with the district level staff and provided one-on-one teacher training. She 
would sometimes guest-teach classes or sit down one-on-one with a teacher to help them overcome technology 
problems. 

How does a grant-related influx of technology in a rural school affect the integration of technology in teacher 
planning and delivery of instruction? 
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Differences in technology use could be observed in teachers. Additionally, the support structure provided 
evidence of how teacher technology integration was being supported. 

It was reported that teachers who had not been using technology started using technology. Different 
participants often talked about one very experienced science teacher, who considered himself a novice when it came 
to computer technology. One teacher said, 

We have two science teachers, who didn’t ever use technology and they had their kids in the 

computer lab this year. I think this may be because they agreed to take the kids to PLATO™ so 

then they have gotten more comfortable and have moved to our Mac lab to do other things. 

When interviewed, the teacher who had limited use with computers related, “Technology, I didn’t like it very 
much-I like it a lot more now, a lot more now.” Examples of how teaching practice changed were seen throughout 
the school. Teachers not involved in the grant benefited from shared information from their colleagues. A non- 
participating coach began using a spreadsheet to keep track of students’ weight training with the help of a grant 
teacher. Also, a social studies teacher changed his attitude about allowing his students to use technology after being 
given the laptop. In an interview he said, “I used to not use computers at all, as far as instruction and stuff like that. 
A lot of time in my research projects that my students would do, I would ask them not to use any computers.” 

Limitations 

There were some difficulties with the grant which affected teacher buy-in. One is the design of the grant, 
and how teachers were required to choose participating classes. Another was the lack of student data to corroborate 
the findings about classroom technology use. 

The Design of the Grant and Teacher Concerns 

Teachers had many concerns over the design of the grant. During the summer workshop, they collaborated 
to develop a way in which one group of students could be involved in a project all day long. Once school started 
they found that administrators had determined the intervention would be implemented in a different way, as related 
by one teacher. 

I don’t think that we will have anything to prove anyway because we were supposed to have just one group 
of kids that we’re tracking and looking for improvement by using this technology and we are supposed to see 
improvement in reading. I would be really surprised if we had five kids in common between the three teachers on 
our team that are doing it, which is going to be statistically nothing. We are not going to be able to do anything. I 
mean this is like, our concern is, we are doing this whole big grant, and we are not going to have these things 
measurable and I don’t think we are. And I don’t think we will have anything that we can draw conclusions from. 

Another participating language arts teacher said “I wish that the ‘be-all and the end-all’ of the success of a 
program did not rest on test scores.” In addition to the teacher’s concerns, the researchers noted there were other 
interventions going on in the school to improve test scores with the same groups of students. Inspiration™ had been 
previously available at the school and teachers had been trained in using it for two years prior to the implementation 
of the grant. Teachers not participating in the grant were using the software with their students as well. 

Impact on Students 

Due to limited access to student information, student achievement was not a focus of the study, but teachers 
were asked for general impressions of student achievement, a measure that we speculated would impact the 
teachers’ attitudes toward technology use. These observations were mixed among the interviewees. A 
nonparticipating teacher stated she was not sure if it had an impact on students because she sees a big change in 
students every year from beginning to end, and could not attribute the changes she was seeing to grant participation 
or not. She did not feel that her choice not to participate disadvantaged her students in any way “Well, I don’t feel 
that my students have been hurting this year, because I feel like I am good teacher, and I do everything I can to make 
sure they can achieve and meet the standards.” 



Implications 

These concerns and observations provide insight for similar implementations in the future. Teachers 
wanted to be included in decis ions involved with the grant and felt that they were overlooked in its design. In 
addition, student-outcomes are hard to measure in such a short period of time. The importance of formative and 
summative assessments is vital in this respect. For example, this grant relies heavily on teacher perception of student 
success as formative assessment, and a SAMS test as a summative assessment. Teachers really have no clear 
measure on whether the student achievement is changing throughout the year, especially related to the state’s high 
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stakes standardized test. Better formative assessment may help with teachers’ understanding of the connection 
between the software and student achievement, and in turn, help them participate in meeting the goals of the grant. 
Those involved in administering the grant program were aware of the hard data that the software programs could 
provide, but there was no structure to use it in a formative manner. 

The laptop program is successful in familiarizing the teachers with technology and increasing their comfort 
levels using the programs. Teachers reported using the technology in new and different ways. Teachers have 
adapted their lesson plans to utilize the technology and integrate it to meet various components of their coursework. 
As mentioned earlier. Heath et al.(2001) suggest two factors that influence teacher attitude change toward 
technology integration: (1) having a willingness to change, and (2) the control structure of the school environment. 

The results of the interviews at this rural school provided evidence of a willingness to change as shown by 
teachers who had not used technology, began to utilize it in their classrooms. One area that might be impeding 
change is the powerlessness that the teachers felt in relationship to the research design. By increasing teacher’s 
ability to input into the research and grant process, this school may be able to increase the number of teachers 
embracing technology as part of their teaching practices. 

While not generalizable to all schools that receive federal grant funding, this case illustrates several ideas 
that are found on the literature about technology integration. For example, like Wang, Johnson, & Pisapia (1994) 
found, providing time and support is important in supporting teachers in adopting new technology. In addition, as 
Reeves (2003) discussed, funding can be a tool to assist a school in complying with new requirements, such as 
legislation. The influx of grant funds into this rural school allowed the teachers and administrators to focus 
considerable attention on complying with NCLB regulations. 
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Developing a Comprehensive Instructional Planning and Improvement Model 

for use in K-12 Classrooms 
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Description 

This PT3 initiative focuses on the development and dissemination of the Instructional Technology, 
Assessment and Action Research (ITAAR) Model, an instructional planning and improvement model for use in K- 
12 classrooms that incorporates instructional design principles, data driven decision-making, technology, classroom- 
based assessment, and action research. This presentation will report on accomplishments from the first of a three 
year grant including the development of the ITAAR model, as well as resulting revised course designs and 
implementation plans. 



Abstract 

Introduction 

With the NCLB legislation of 2001 requiring all teachers to measure student progress using tests aligned to 
state standards and holding schools accountable for student achievement, all teachers must be familiar with not only 
high stakes assessment, but also using data from ongoing assessment to make instructional decisions. The 
Instructional Technology, Assessment and Action Research (ITAAR) Project (PT3 funded 2003) attempts to address 
this by developing an instructional planning and improvement model that incorporates the use of advanced 
technologies for, data driven decision making and action research into the assessment and instructional processes for 
the improvement of student achievement. 

The ITAAR project is a curriculum redesign effort with several supporting initiatives including the 
development of the ITAAR Model, the integration of the model in course work, the training of teams of faculty, 
clinical supervisors, and cooperating teachers and preservice teachers. This first year of the ITAAR Project focused 
on developing the Model and training faculty teams. 

Building The ITAAR Model 

At the core of the ITAAR Project is a consortium that includes the Colleges of Education and Liberal Arts 
and Sciences at a Major Midwestern University and five school districts including one large urban district and four 
smaller districts. University faculty, clinical supervisors, and cooperating (mentor) teachers worked collaboratively 
to develop the ITAAR Model that introduces prospective teachers to instructional technologies, continuous 
classroom assessment, and action research. The ITAAR Model includes basic and advanced technologies, an 
instructional design model, and purposeful accumulation of evidence of learning and achievement through handheld 
technology. One of the primary challenges in conceptualizing the model is its usability. Therefore, readily available, 
authentically designed and practical resources were integrated. Resources include techniques in eportfolios and 
action research using classroom data, the INTIME instructional modules, specific methods of instruction, and a 
newly developed collaboration technology used to track clinical experiences and network preservice teachers with 
each other and faculty teams. 

The ITAAR Model was introduced to faculty teams in a four-day intensive training institute for paired 
teams of faculty, clinical supervisors and cooperating teachers. Courses in technology and assessment were revised 
to include instruction for preservice teachers in the components of the Model. These teams will continue their work 
in curricular revision throughout the year. 

Core Components of the Model 

Assessment. By using standards aligned assessments, and efficient data collection methods and analyses, 
time -strapped teachers can implement proven instructional design strategies aimed at facilitating student 
achievement. The standards-based, objective nature of the assessments will facilitate the connection between the 
results of the data analysis and the continuous improvement of instruction stressed in the Data Driven Decision 
instructional technology model. With the increased demands for accountability, teachers must be able to 
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demonstrate student achievement through high stakes and alternate assessments, through report cards and IEP 
reporting systems, and must have a built in structure for not only collecting the data, but for analyzing, reporting, 
and using the data to make informed instructional decisions 

Handheld Computers. Fortunately, advances in technology through the use of handheld PDA’s or other 
technology can make this feat much easier. The portable nature of these technologies allows these “decisions” to be 
made “on-the-fly,” with coherency and transparency. As the knowledge base of handhelds in education is growing 
by leaps and bounds, one area that needs definite attention mirrors the goals of this project: using handhelds for 
assessment and improvement of instruction. 

Instructional Technology and Action Research Since we believe that teachers need to assess and discover 
for themselves the effectiveness and the problems of technology integration, action research needs to be encouraged 
as a methodological tool to help improve instruction. We see action research as a tool for enhancing learning 
environments by involving teachers and their students in focusing on solutions to instructional problems. 

Electronic Portfolios By tightly integrating electronic portfolio concepts within the assessment driven 
instructional design model, itself integrated at the college and university level, building on the successes of previous 
projects, and remaining dedicated to preparing our pre -service teachers to engage in effective, standards based, 
decision making assessment practices, this program will lead to more targeted and efficient instruction, enabling K- 
12 students to achieve their potential. 



Continuing the Project 

As adoption and implementation of the model continues, faculty will evaluate the Model for many factors 
that will contribute to the continuous improvement of the Model and the learning materials used to communicate it. 
These factors include the overall usability of the model, the degree to which the Model is consistent with underlying 
theories of research, assessment, and instructional technology, and the degree to which the Model is appropriate, in 
research and application, for the specific content areas represented by faculty. 

In future phases of the project, content and methods courses for preservice teachers will be revised to 
include the use of components of the Model. Prospective teachers will be further supported in building their 
capacity in employing advanced technologies to interpret, analyze and incorporate student achievement and testing 
data into the instruction process by attending Preservice Teacher Workshops that repeat, reinforce, and expand on 
technology and assessment components of their regular, program courses. Independent Learning Modules, available 
on CD or on the Web, elaborating on the Model will provide as -needed support for faculty, clinical supervisors, and 
cooperating teachers. Preservice teachers will be able to access specific Modules to support their clinical 
experiences and subsequent teaching. 
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Promoting Technology Integration in Teacher Education Through Faculty 

Development 

Corenna Cummings 
Deborah Kalkman 
Jason Underwood 
Northern Illinois University 

Description 

This presentation will share several successful faculty development strategies for effectively integrating 
technology into teacher education curricula, evaluation findings from faculty technology integration experiences, 
and provide examples of quality technology integration projects created by teacher education faculty. The 
experiences of participating faculty will be shared through their own evaluations of their projects, descriptions and 
products of the projects, perceived and actual impact of their efforts, as well as reflections on professional growth 
and curricular change. 



Abstract 

Recent studies indicate significant progress had been made toward providing K-12 schools with 
technology. Despite efforts of some school districts to in-service teachers on the effective integration of technology 
into instruction, there continues to be a gap between the availability of hardware and teachers’ ability to effectively 
use it instructionally to promote student learning. Even when school districts have succeeded in integrating 
technology into their curriculum, graduates of teacher education programs are, in general, ill-prepared to do 
likewise, thus requiring school districts to provide this training for their new hires. It becomes imperative that 
teacher education programs address the integration of technology in their teacher certification programs. Faculty 
modeling the integration of technology in courses and requiring students to use technology to complete assignments 
and integrate it as they develop their own lesson plans and instructional materials are two vehicles by which pre - 
service teachers may learn to effectively and meaningfully integrate technology into the curriculum; however, 
university faculty, instructors, and pre -service teacher clinical supervisors often lack the technical and pedagogical 
knowledge and skills to be effective models. 

To address these important problems, the Colleges of Education and Liberal Arts and Sciences at Northern 
Illinois University (NIU) have partnered to create the CENTTER Project: a Collaborative Enterprise to Integrate 
Technology into Teacher Education Reform. The CENTTER project is funded in part by a 2001 U.S. Department of 
Education PT3 Implementation Grant, and is designed to promote pre -service teachers’ ability to effectively and 
meaningfully engage students in learning by immersing them in technology rich learning environments in college 
classrooms and clinical experiences. To facilitate the creation of these technology rich learning environments, the 
CENTTER Project has adopted successful faculty development strategies, offered incentives for faculty learning and 
integrating technology into their curricula, and provided support for faculty learning and teaching with technology. 
These initiatives have facilitated many exciting, effective faculty technology integration projects, with diverse 
technologies, strategies, and outcomes. 

Two primary faculty development strategies used are a faculty course. Integrating Technology into the 
Curriculum (ITC), and a network of topical workshops. ITC, a dynamic technology integration course for college 
faculty, instructors, and pre -service teacher clinical supervisors, provides participants with the knowledge and skills 
needed to effectively integrate technology into their courses and students’ clinical experiences. ITC course topics 
include creating multimedia rich presentations, designing and developing websites, and web-based learning 
experiences, creating electronic portfolios, using handheld computers in the classroom, as well as critical issues such 
as accessibility and ethics. The network of topical workshops include workshops offered by project staff, often 
customized to faculty need, as well as workshops offered by the College of Education and University through such 
organizations as the Faculty Development and Instructional Design Center (FDIDC), the Office of Instructional 
Assistance (OIA), and the College of Education’s Technology Support Center. 

To encourage faculty to learn and teach effectively with technology. The CENTTER Project has developed 
a system of incentives including “Mini-Grants” and financial support for dissemination of information related to the 
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goals of the project. “Mini-Grant’' proposals are accepted from faculty each semester with award amounts ranging 
between $500 and $2000 for pedagogically sound, attainable technology integration projects. Faculty are able to 
apply for expense reimbursement related to dissemination efforts including presenting at technology and research 
conferences. 

As faculty engage in meaningful technology integration experiences, they often have needs they cannot 
satisfy on their own. To address these needs, the CENTTER Project has developed a comprehensive network of 
financial, technical, pedagogical, and equipment related support. Financial support is available through the purchase 
of software, the services of graduate assistants, and the purchase of hardware. Cooperative Theme Groups (CTG), 
composed of faculty with similar technology integration goals and projects, serve as a source of technical support, in 
addition to one-on-one support by Project staff, and technical advice available in workshops and in ITC. CTGs are 
also a source of pedagogical support as are Project staff members. With budgetary constraints in full force, many 
faculty benefit from our loaner equipment support, with items such as digital cameras, video cameras, handheld 
computers, laptops, projectors, and whiteboard capture devices. 

These strategies, learning opportunities, incentives, and support have enabled faculty in the College and 
University to create and implement many exciting and meaningful technology projects. The technologies used are 
diverse; including web based learning experiences such as web sites, WebQuests, and content delivery on the web, 
digital video, handheld computers, mind-mapping software, and multimedia presentation tools. The audiences are 
primarily early childhood, elementary, secondary education, and special education pre -service teachers, in the 
Colleges of Education and Liberal Arts and Sciences. These projects have had a tremendous impact, reaching 
thousands of students, encouraging curricular change, and promoting faculty professional growth. Often, the pre- 
service teachers conduct their own projects and adopt successful technology integration strategies that have been 
modeled by their instructors. 
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Improving Open Online Content Development for K-12 Education 

Vedat G. Diker 
University of Maryland 

Abstract 

The study summarized in this paper explored policies for improving open online content development 
projects with respect to growing the development community and improving the quantity and quality of materials 
developed. The study made use of dynamic feedback simulation and interviews with the members of an established 
content development community that specializes in K-12 instructional materials development. 

Open Online Content Development 

Open online content development is a collaborative authoring paradigm which has its roots in the free/open 
source software development movement (Keats 2003). The success of major open source software development 
projects such as Linux (Torvalds 1999), Apache (Fielding 1999), and Perl (Wall 1999) added credibility to the open 
online content development paradigm and encouraged the application of the paradigm to content development in 
other areas. 

Open online content development takes place in online communities comprised of voluntary contributors 
and users. Contribution is mostly open to all with little or no barriers to entry. The information products or content 
collections developed by the contributors can generally be used freely by anybody that has Internet access. The open 
online content development paradigm can be used to deliver highly accessible instructional materials to a wide 
audience of educators and students. 

Developing open online content development communities poses certain challenges, such as motivating 
contributors to participate, attracting users, and building content collections that are rich in both quantity and quality. 
Furthermore, most of those challenges are interconnected. For example, literature suggests that an important 
motivating factor for contributors is knowing that a high number of users benefit from their contributions to the 
content collection (Kollock 1999). Also, the number and talent level of contributors are critical determinants of the 
quantity and the quality of the content developed. Accordingly, focusing on a single challenge in an open online 
content community generally does not improve the overall growth and performance of the community. 

Policies for Improving Open Online Content Development 

The study summarized in this paper explored policies for improving open online content development 
projects with respect to growing the size of the development community and improving the quantity and quality of 
materials developed. The study made use of dynamic feedback simulation and interviews with the members of an 
established content development community that specializes in K-12 instructional materials development. 

The first step of the study involved the development of a dynamic feedback simulation model that 
represents the causal relations among the determinants of success in a typical open online content development 
community. More specifically, the simulation model was conceptualized as a representation of a hypothetical open 
source software development community. The model was based on implications derived from three streams of 
literature: 1) theoretical literature on online communities, 2) theoretical and practical literature on open source 
software development, and 3) literature on dynamic feedback simulation models of software project management. 
The simulation model was tested extensively for internal validity (Diker 2003, pp. 249-394). 

A series of policy analysis simulations were performed on the model. Policy analysis simulations involve 
running the model under different values of parameters that can be controlled by the decision and policy makers of 
the real system that the model represents. The analysis of the policy simulations identified the chief underlying 
policy problem in open online content development as the tension between building new content and improving the 
quality of existing content. This problem manifests itself as a barrier to improving both quantity and quality of 
content simultaneously beyond a certain level (Diker 2003, pp. 366). 

Four main policy options for improving the quantity and quality of the content collection without 
undermining either performance measure were tested on the model: 1) filtering new materials, 2) reviewing and 
editing existing materials, 3) selecting new inexperienced authors, and 4) coaching existing inexperienced authors. 
Combinations of these pure policy options were also tested. The two most promising policy options that emerged 
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were selecting new inexperienced authors, and a combination of coaching and reviewing/editing (Diker 2003, pp. 
327-366). 

These findings were tested against the mental models and the observations of an actual open online content 
development community. The specific community studied was a group of teachers and researchers who develop and 
disseminate instructional materials for introducing system dynamics concepts to K-12 students. The system 
dynamics K-12 instructional materials development community has gathered around four main organizations or 
groups: two of these are non-profit organizations propagating systems thinking and system dynamics in K-12 
education, and the other two are research and practice groups working on developing instructional materials for 
introducing system dynamics concepts to K-12 students. Ten leading members of the community were interviewed, 
and the interview data were analyzed qualitatively (Diker 2003, pp. 395-397). 

The analysis of the interviews supported the findings of the model simulations that the fundamental policy 
problem in the community with respect to content development was the tension between building the quantity and 
the quality of the content in the collection. Most interviewees stated that they had observed filtering, 
reviewing/editing, and coaching policies implemented in their community. Three interviewees suggested that they 
had observed a selecting policy implemented covertly in addition to the other three options. However, more than 
half of the interviewees argued that an overt selection policy would be detrimental for their specific community. 
They mentioned the welcoming culture of the community to be main reason why such an overt policy would not be 
desirable and beneficial. Most interviewees suggested coaching as the most beneficial policy in the long run. Some 
of these interviewees suggested that a combination of coaching and reviewing/editing would work best (Diker 2003, 
pp. 412-435). 
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Abstract 

College and university faculty face many demands on their time: research, teaching, service, committees, 
family, and other obligations. One of the major challenges faced by instructional support personnel at colleges and 
universities is to get their training efforts entered into the faculty members’ busy calendars. This paper looks at a 
new option for delivering training in technology and pedagogy through streaming video. The project described 
involved faculty members sharing their technology-related projects and instructional strategies with other faculty 
through online video, which could be viewed either live or at the faculty members’ convenience. Faculty response 
to the program is described. 



Introduction 

Instructional support staff at postsecondary institutions confront challenges in their efforts to deliver 
training to faculty members. The faculty members that they support face numerous demands on their time: teaching, 
research, conferences, office hours, and a seemingly endless number of meetings. Not to mention family and social 
obligations. With each of these areas taking time out of faculty members’ schedule, little is left for developing new 
technical and pedagogical skills and strategies. 

In our school, we have a large number of faculty in this situation. They are generally positive and 
enthusiastic about professional development, but find it difficult to find the time in their overextended calendars to 
attend workshops or even to come to our office for individual consultations. We constantly meet up with instructors 
in the halls who say, “I’ve been meaning to come to you to talk about a certain project, but I just haven’t had the 
time.” Because of this, the staff in our instructional support office have long been looking for new ways to deliver 
training to our faculty. 

Recently, one of our staff noted the large number of faculty members lined up in front of the snack cart in 
our building’s atrium, purchasing their lunches. These faculty members were going to take a chicken sandwich or a 
salad back to their office to eat while sorting through email, surfing the web, or listening to online radio. It struck 
the staff member that this might be an ideal opportunity to deliver some training to the faculty members. 

A New Way to Reach Overextended Faculty 

From this insight, the idea for our online streaming video series came about. We envisioned faculty 
members watching short, targeted video presentations while eating their lunches at their desks. The faculty all had 
Ethernet connections to their computers, soundcards and speakers, and Realvideo installed by default, so hardware 
and software would not be a problem. We had a distance education room set up with cameras, and a call to the 
streaming media people at our university comp uting services department was all that it took to arrange for the first 
video session. 

What would the content of these streaming video broadcasts consist of? From past experience, we knew that faculty 
members often felt somewhat isolated, in that they seldom had the opportunity to learn about what other faculty 
members, especially outside of their departments, were doing in terms of pedagogical and technological innovations. 
We chose a number of faculty members that we had worked with and that were doing exciting things in their on- 
campus and online classrooms, and arranged a schedule of video sessions with them. 

We chose every other Wednesday at noon for the live broadcast, with an archive version of the sessions available 
almost immediately afterwards. This way, if faculty members could not watch the live broadcast on Wednesday, it 
would still be available for them to view at a later time. 

Presentation Format 

The sessions were one half hour to forty-five minutes in length, so that the faculty member viewing the 
program would not have to devote his/her entire lunch hour to the program. We wanted to incorporate interaction 
from our faculty viewers, so we had an Internet-connected laptop operating at the site, so that we could collect 
emailed questions as they came in. We considered instant messaging as an option, also, but since many faculty 
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members were unfamiliar with this type of software and would not have it on their desktops, we decided to go with 
email, a delivery method that they were comfortable with and that did not require any new software. To encourage 
participation in this way, we also offered free coffee mugs to anyone whose question was read on the air. On a 
couple of occasions, we had a live audience in the distance education studio where the sessions were broadcast from, 
but most of the time, it was just the presenter, the host and the technical support personnel. The choice was up to 
instructor preference. 

The director of our office and the head of training for our school, both of whom have had extensive 
experience in giving workshops and training sessions, alternated in the role of host for the broadcasts. The faculty 
members were encouraged to engage with the host in an informal discussion about their topic, rather than give a 
typical dry conference presentation. We encouraged them to use Powerpoint slides, but told them to limit the 
number that they presented. We didn’t want the content of these presentations to be bound to getting through a large 
number of slides. 

Presentation Content 

As stated previously, one of our primary goals in developing this video series was to provide faculty with a 
venue in which they could share their pedagogical and technological skills and practices with each other. This sort 
of collegial discourse is often lacking at our universities, especially across departments. Most of our faculty have 
little time, except for events like the yearly faculty retreat, to reflect upon and share what they are doing in their 
classrooms. Too often, this sort of discussion is not viewed as scholarly activity, and thus is not as highly valued as 
perhaps it should be. Also, some faculty may find it awkward to seek assistance in their teaching efforts, especially 
if they are, like our professors, in a school of education. Doesn’t having a Ph.D. in Education make you an expert in 
teaching? They may feel that they shouldn’t need any additional development in this area, and be hesitant to appear 
in public, in front of their peers, at a face-to-face session. This new delivery method allows instructors to “lurk” — 
getting the information that they need without having to be seen doing so. 

The tone of these sessions was collegial and informal, less like a lecture and more like the conversation that 
might happen between sessions at a professional conference, where a faculty member shares his/her “best practices” 
with a colleague in a friendly manner. The host engaged the guest faculty member in conversation about their 
experiences related to the topic, and questions from the audience, either online or in -person, were answered. 

The topics of our initial series of online streaming video “talk shows” were: 

• Web-based Search Strategies 

• Designing Online Course Components 

• Creativity and Technology in Education 

• Plagiarism 

• Teaching with Web-based Discussion Forums 

• Web Accessibility 

• Blended Learning Options 

• Using Simulations in Teaching 

These topics were based on our knowledge of the faculty, as instructional consultants who worked with them 
every day, and of their current research interests and development projects. We advertised the upcoming broadcasts 
through our email mailing list, which went out to all faculty members. We also invited associate instructors 
(graduate students with teaching positions) through email to “tune in” to the sessions. Also, individual faculty 
presenters notified their students and colleagues about the presentations, which gave us, at times, an international 
audience. 

Since streaming video has worked so well for us for presenting our “E-Brownbag” series of faculty 
workshops, we have now broadened its use. We now use it to present archived versions of our technology training 
sessions, such as recent workshops on the topics of intermediate Dreamweaver for educators, digital video 
production, and desktop publishing. The system is also being used to deliver colloquia for our online Master’s 
program in instructional technology. 

Technology Choices and Developments 

We had the benefit of previous university investment in Internet and distance education technology when 
we planned out the technical side of these presentations. When the new School of Education building was built, in 
1998, distance education rooms were created with the latest in video networking technology, and the university has 
continued to upgrade the facilities. We were able to use one of the distance education rooms as the studio for our 
live webcasts. This room has numerous microphones, video cameras, and large-screen monitors, along with a 
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computer, VCR and document camera for presenting media during the session. There is a control panel for 
switching cameras, and for choosing the computer input for displaying Powerpoint slides and web pages, and a mute 
button for times when you don’t want the audio to be broadcast. During the live broadcast, the signal went out over 
a high-speed Polycom video network, and was captured by a downstream RealVideo streaming server, which 
created a stream over the Internet that anyone with a current Realvideo player could access. 

For the archived version of the sessions, at first we just provided interested faculty with a webpage that 
included a link to the Realvideo file for them to view. We also set up a website that included the Powerpoint file for 
downloading. Faculty could view the video file using Realplayer and see the accompanying slides using 
Powerpoint. However, the lack of sync between the two was less than ideal. We researched various options for 
presenting the two media together. There are any number of commercial systems, most of them targeting business 
clients, that offer this sort of online lecture hall or meeting room, among them Microsoft Presentation Broadcast, 

Real PresenterOne, Macromedia Breeze, Jet Stream Jet Manager, Intercall MShow, and sofTV.presenter. However, 
for technical and/or cost reasons, none of them met our needs. We were locked into two technical choices: Microsoft 
Powerpoint, as that was the presentation format familiar to faculty, and Realvideo, as that was the format supported 
by our university’s streaming servers. We also had the additional constraint of minimal to no budget for the project. 
For these reasons, we chose to develop our own system, which we christened the Virtual Internet Presenter (VIP). 

The VIP system is a frames-based system, with the Realvideo plug-in embedded in the top left frame, and 
the Powerpoint slides (converted to JPEG images) in the larger, right frame, covering roughly 75% of the window. 
The space under the video display is used to provide a listing of the slides. Clicking on any of the items on the 
listing will take you to that slide and move to that point in the Realvideo presentation. Similarly, as the presentation 
progresses, the current slide is designated by a triangle next to the slide name. 

The program was written by a talented colleague of mine, Larry Campbell, using Perl (mod-Perl to be 
specific), Javascript and VBScript. The program went through many iterations as we continually tested it and 
suggested refinements and additional features. He developed an administrative interface (figure 1) for the system 
that allowed us to input the video files and Powerpoint slides easily, and to synchronize them. The interface was 
connected to a web-enabled database that created an attractive and informative menu page (figure 2) that listed the 
available videos and provided information about the content and the presenter, along with a thumbnail photograph of 
the presenter. The system was built upon the underpinnings of a Linux operating system, Apache web server 
software and a MySQL database. We chose this setup for security and reliability reasons, as well as the fact that the 
open-source software was free to use. 

One major challenge in the development of the tool was to get it to work with the majority of web 
browsers, including the various releases of Internet Explorer and Netscape. A further difficulty was getting the 
system to work with Apple Macintosh browsers, including Apple’s own Safari. To date, this has not been fully 
accomplished. It is possible to view the presentations on Apple Macintosh computers running Safari, Internet 
Explorer or Netscape, but the synchronization between the video and the Powerpoint presentation does not work, 
and it is necessary for the viewer to navigate through the slides manually. We hope to eventually fix this 
challenging problem. Numerous options have been attempted, unsuccessfully, and it may require waiting for Apple 
to update its operating system software or for a new version of one of the browsers. 

The VIP logo was designed, in Flash and Realvideo versions, by Jung Won Hur, one of our office’s 
graduate assistants. It is based on the countdown leader that is traditionally shown before 16mm and 35mm films 
are projected. Other than that, the graphic interface of the system is plain but functional. While no formal usability 
testing was conducted, the system is so simple that the end users have not had any complaints. The success of the 
user interface relies on the fact that it relies upon familiar technology — the web browser and the Realvideo player. 
The vast majority of our faculty are familiar with both. 

We were fortunate that most of the viewers of the video would be working at their School of Education 
office workstations, with a known technical configuration. Some of our users needed to have Realplayer installed or 
updated to the latest version, but that was the only technical support required. We did receive some complaints of 
dropouts in sound or video due to network congestion or server load, but these were short-term and temporary 
problems. Sometimes it took awhile for a presentation to get started, due to the way Realplayer buffers a certain 
portion of the file before starting to play. 

Faculty Response 

We were very pleased with our viewership for the series. Our school has just over 100 full-time faculty 
members, and our live broadcasts reached 20 to 80 viewers, depending on episode. The archived VIP versions of 
the presentations have had from 80 to almost 300 views. This can be compared with on-campus workshops where 
we are lucky to have 10 faculty members show up. And, while the primary audience for the presentations is our 
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faculty, the website is open to the world. The VIP system has had visits from Australia, Canada, Austria, the United 
Kingdom, Taiwan, Philippines, Korea, France, Japan, China, the Netherlands, Singapore, India, Spain, Turkey, 
Greece, Cyprus, Italy, Malaysia, New Zealand, Hong Kong and Malaysia. These international numbers are not large 
(generally under 20 visits) but the series has not been advertised anywhere outside of our school (except for faculty 
members contacting their colleagues about the presentations). Overall, the VIP video system has streamed over 
3300 video sessions. 

We have received many compliments on the quality of the series from faculty and associate instructors, as 
well as from the administration. Faculty members have commented about how convenient it is, being able to view 
these training sessions at any time from their office or (in many cases) home computers. The fact that the university- 
sponsored technology workshops, offered by the central computing services, are always offered in a distant building, 
makes it hard for them to attend. And even the workshops that we offer in the building often conflict in time with 
departmental or committee meetings, office hours and other obligations. They also appreciate being able to 
randomly access any part of the presentation, and being able to repeat or skip sections. The fact that they can 
multitask — listening to the audio of the presentation while skimming through email, eating lunch, etc. — is also 
appreciated. Most faculty take notes at our workshop, but having access to the entire audio and video of the 
workshop, as well as the slides, makes it easy to check on something that might not be clear in one’s notes. 

In feedback forms, we received comments such as “I would love to see all kinds of seminars and topics 
archived as the OnCourse seminar was !” Another appreciated the fact that “it could be taken in the comfort of my 
office & at my convenience.” The only complaints we received about the series were due to technical issues, as 
mentioned above, not about the content of the video presentations. 

Further Developments 

In the future, we plan to further develop the VIP system in several ways. The top priority is to ensure full 
Macintosh compatibility. While a minority of our faculty use Macs (only 22 out of 1 12), we still want to be able to 
reach every faculty member with our training activities. We are hoping to find a programmer who is experienced in 
the OSX environment to help us with this issue. Another area for further development is the administrative 
interface, which has basic functionality but lacks features such as the ability to view the slides that you are adding to 
the presentation in the same window as the video It is also lacking in visual appeal, and more importantly, 
documentation. There has been preliminary talk about developing a commercial version of the tool but this is just 
speculation at the moment. Right now, we are focusing on meeting the needs of our local faculty members. 

Conclusion 

Based on our successful experiences with this program, we recommend that other postsecondary 
instructional support offices consider streaming video as a new option for delivery professional development in 
technological and pedagogical skills and strategies. This delivery medium engages faculty where they are, at a 
convenient time, using technology that they are already familiar with. A program such as ours also provides an ideal 
way for faculty to share their innovations with each other. 
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Faculty Beliefs about Teaching with Technology 

Pamela Ferguson 
Georgia State University 

Introduction 

The rapid growth of instructional technologies has provided college faculty with a vast array of educational 
resources and learning opportunities (Grasha, 2000; Kagirna & Hausafus, 2001). As described in the relevant 
literature addressing this issue, there are several potential advantages in utilizing instructional technologies to enrich 
the educational experience. For example, McKenna, Avery and Schuchardt (2000), identified numerous potential 
advantages from integrating technology into instruction, including increasing the opportunity for individualized 
learning by students, offering a new way of thinking and communicating for both students and professors, 
expanding the emphasis on problem-solving, and enabling the learning of higher-level skills including, embedding 
learning in relevant contexts, critical thinking, goal-setting, planning and self-monitoring. Kagima and Hausafus 
stressed that incorporating technology into teaching methodology allows educators the flexibility to tailor 
educational resources to accommodate diverse learning styles, cultural differences, skill levels, motivations, 
disabilities, and educational objectives. 

Additionally, DeSieno (1995) noted three advantages of integrating technology in the teaching-learning 
process. The first notable advantage is that, the learning environment is transformed and information becomes more 
vivid, instructive, and focused upon the immediate needs of students. The second advantage noted by DeSieno was 
that, the use of technology can encourage students to become more responsible for their own learning. A final 
advantage is that technology can assist students in learning the fundamentals of a specific subject, thereby freeing 
faculty to concentrate on working with students in mastering the more complex aspects of their courses. 

Despite the existence of these advantages, relatively little attention has been focused on how teachers’ 
views of learning affect the ways they use technology in the classroom (Futon, Torney-Purta, 2000). This research 
study attempts to examine this question by focusing on the relationships among pedagogical beliefs and teaching 
styles, as well as examining the relationship between faculty beliefs about teaching with technology and teaching 
strategies. 

Teaching styles and technology usage 

Teachers have preferred teaching styles, with which they are comfortable and to which they can easily 
revert. While some researchers equate teaching style with teaching method or technique, Heimlich and Norland 
(2002) emphasize that most researchers define teaching style as a predilection toward teaching behavior and the 
congruence between an educator’s teaching behavior and teaching beliefs. Numerous models of teaching styles are 
available, including the five styles outlined by Grasha (1996), which are expert, formal authority, personal model, 
facilitator, and delegator. Each teaching style is associated with particular teaching roles, attitudes, and teaching 
strategies. Furthermore, several authors assert that effective teaching requires a melding of teaching and 
technological expertise, meaning that teaching and technology are becoming increasingly co-dependent (Pierson, 
2001; Woodbridge, 2003). Pierson stresses that technology expertise does not always relate to teaching expertise, 
nor does good teaching always translate into effective technology use. Jacobsen (2000) advocates the position that 
learning how to teach with technology requires imagination, intellect, and creativity. 

Learning theory and technology’s role 

The teacher-centered classroom illustrates the behaviorist approach to teaching and learning. According to 
behaviorist learning theory, learning is a change in behavior due to experience or environment. Experience is the 
product of building associations between the occasion on which the behavior occurs and the behavior itself. Under 
behaviorist theory, desired responses are rewarded and thereby reinforced, which leads to their repetition. This 
model of instruction fits within the traditional transmission model of teaching and learning. Transmission pedagogy 
derives from a conventional theory of learning in which understanding occurs from carefully planned instruction on 
a narrowly -defined skill or content topic and guided practice on questions related to that topic. Fulton & Torney- 
Purta (2000) define the transmission model as a pedagogical approach in which teaching is telling and learning is 
listening. 

Constructivist views of learning focus on knowledge construction. Under this theory of learning, 
knowledge is not passively accumulated, rather learning results from active cognizing by the individual, using 
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processes such as abstraction, reflection, and the creation of knowledge structure. Proponents of this theory 
emphasize that knowledge acquisition and meaning making cannot simply be transferred from one individual or 
group of individuals to another, but rather, the construction of knowledge and the making of meaning are 
individually and socially constructed. Thus, individuals use active processes to create their knowledge and meaning 
of the world. The foundation of constructivism is the field of cognitive learning theory. Researchers such as Piaget 
and Vygotsky, extrapolated their findings in cognitive development to encompass influences from the learner's 
environment, including the role of the instructor and other social, cultural, or historical factors. Technology use 
stemming from this framework emphasizes inquiry, simulations, and the development of on-line learning 
communities (Fulton & Torney-Purta, 2000). Constructivists argue understanding cannot be transmitted, nor does 
skills -practice result in understanding that can be automatically applied, as needed. Instead, constructionists believe 
effective teaching involves creating environments in which students take mindful effort towards developing their 
understanding and have opportunities to learn how to apply their knowledge and when to do so. Computers can be 
effectively utilized within instruction based on a constructivist model of learning. For example, presentations to a 
critical audience, integrating different perspectives in a report or multimedia document, or examining contrary 
assumptions using a spreadsheet model. 

Faculty type and technology usage 

Hagner (2001) identified four types of faculty, first-wave (self-starters), second-wave (traditionalist), third- 
wave (careerists), and fourth-wave (reluctants). The following are characteristics of first wave faculty: (a) they 
work in more technologically supportive environments; the impetus for their enterprise is internal, (b) they share a 
strong interest in bettering the quality of the education delivered and the learning produced, (c) they possess 
sufficient expertise to give them the confidence to proceed, (d) they are not motivated by standard academic 
incentives, and (e) their expressed disappointment may influence the extent to which they will continue their 
integration and perhaps more importantly, share their positive experiences with their colleagues. Characteristics of 
second-wave faculty include: (a) fear of the unknown, (b) “If it Ain’t Broke...”, (c) “We’re all alone in this 
togethef ', (d) “Know Thyself' - Adoption to new teaching environments represents a major commitment on the part 
of the faculty member to re-evaluate their own personal approach to learning, and (e) they like the sexy technology, 
but fail to seek out new ways to use hardware or software to expand their teaching repertoire. Third-wave faculty 
are the type that will emerge when universities change their reward structures in the tenure and promotion process; 
they are the teachers who see adopting technology as a means of advancing their professional careers. Fourth-wave 
faculty are either computer illiterate or those who firmly believe in the superiority of traditional models of learning. 

Purpose 

The purpose of this study was to examine relationships among pedagogical beliefs and teaching styles, as 
well as examining the relationship between faculty beliefs about teaching with technology and teaching strategies. 
The following served as guiding questions: 

1. What is the relationship between personal beliefs about integrating technology and teaching styles? 

a) How do instructional technologies fit into a teacher’s philosophy of teaching? 

b) What do teachers see as the major influences on beliefs about teaching and technology 
integration? 

2. Do instructional technologies play a significant role in teachers’ instructional strategies? 

a) How do teachers use instructional technologies in their instruction? 

b) What teaching strategies do teachers find effective in integrating technology into instruction? 

c) How do teachers’ beliefs about their teaching practices correspond to their beliefs about how 
technology should be used in the classroom? 

Methodology 

Research design paradigm 

This study utilizes a qualitative research paradigm and employs a case study methodology to examine 
relationships among pedagogical beliefs and teaching styles, as well as examining the relationship between faculty 
beliefs about teaching with technology and teaching strategies. The methodological design was chosen based on the 
alignment between the research questions and the data collection methodology. Law, Stewart, Letts, et. al (1998) 
state that qualitative research designs seek meaning and understanding and usually involve the in-depth exploration 
of a topic, emphasizing seeking information from the people who are involved. Thus, case study methodology 
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allowed the researcher to interview faculty members in their office environments providing the researcher with an 
insider’s view into the complexities of a faculty member’s natural habitat of learning and teaching. 

Establishing rigors of the study 

To increase internal validity, the following strategies were employed: triangulation of data, member 
checking, peer examination, participatory modes of research, and clarification of researcher bias (Creswell, 2002). 
Triangulation of data was accomplished through collecting data from multiple sources including interviews, field 
notes, and document analysis. Member checking involved returning an interview transcript prior to analysis to the 
participant to obtain their validation and approval. Participants were encouraged to make modifications, additions, 
or deletions to the transcribed record. A research peer debriefing team of doctoral students (n=4) served as peer 
examiners. Additionally, the researcher’s beliefs and practices concerning whether the presence of instruction 
technology in the classroom is associated with variations in teaching styles and strategies, may potentially shape the 
way collected data is viewed, understood, and interpreted therefore a field log and a reflexive process was utilized to 
record researcher personal values, assumptions, and biases. 

Bounding the study 

The study was conducted on the campus of a small liberal arts college (SLAC) founded and located in the 
southern United States. The college is located in a rural Southern community. The institution enrolls over 1 ,300 
students at two locations, with a main campus enrollment of 800 students and an off-campus location enrollment of 
approximately 500 students. Originally accredited as a junior college in 1927, the institution transitioned to a four- 
year institution in 1992. 

Participants 

Creswell (2002) argues in order to best help the researcher understand the problem and the research 
questions, participants should be selected purposefully. Therefore, a purposeful sample of information-rich key 
informants representing a range of disciplines (i.e., three business participants, one math participant and one 
behavioral science participant.) was drawn to obtain a more in-depth picture of teaching philosophies and beliefs 
about teaching with technology. The select group was diverse in terms of technology access, as well as student 
population. The participants consisted of four males and one female. Three participants held the academic rank of 
Associate Professor; one held the academic rank of Assistant Professor and one held the academic rank of Professor. 
Two participants had six to ten years of academic experience while three participants had more than 15 years 
experience. 

Data collection strategies 

Data collection occurred between January 2004 and May 2004. Data collection instruments included a 
field log, survey instrument (Appendix A) that included demographic information, questions about teaching 
philosophy, and specific technology us^e, and semi -structured face-to-face interviews (Appendix B). To assist in 
the data collection phase, the researcher utilized a field log to record details of the researchers’ thoughts, feelings, 
experiences, and perceptions throughout the research process (Creswell, 2002). 

The interview procedure was semi-structured and open-ended in nature in order to be responsive to 
emergent topics and themes. Open-ended interview questions were used to provide rich qualitative data about the 
influence of instructional technologies on teaching styles. The primary objective of the interviews was to record, 
analyze, and interpret the individual college faculty member’s experiences, opinions, and perspectives with regard to 
instructional technologies, teaching philosophy and strategies. The initial interview took approximately 45 minutes 
to complete and was conducted with the aid of an interview guide. The interview guide addressed teaching style, and 
methods for using and integrating technology in teaching. Interview times were arranged based on the participant’s 
request and the researcher’s availability. Interviews were audio recorded and transcribed verbatim. Prior to analysis, 
the interview transcripts were returned to participants for validation and approval. Participants were encouraged to 
make modifications, additions, or deletions to the transcribed record. 

The anonymity of the interview participants in this research and subsequent potential publications was 
protected using the following methods; the use of a pseudonym; and the use of audio recordings, transcripts, and 
observational notes which were maintained in the researcher’s secure files throughout the research process. All 
artifacts were coded, so that no personal identifying information was visible and audio recordings were listened to 
by the researcher. 

Data analysis 
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Data analysis for the interviews was inductive and the overall data analysis strategy employed a constant 
comparative methodology (Glaser & Strauss, 1967). Glaser and Strauss identified four stages of constant 
comparative analysis: compare the incidents, compare the incidents within each category, reduce the categories to a 
smaller number and begin to develop a framework for understanding the data and begin thinking in terms of theory 
based on the coded data. 

The interview data was scanned for similarities and anomalies. Initially, a list of repeated words, phrases 
and ideas were coded in the margins of the transcripts. The data was analyzed into themes suggested by the data, to 
identify units of information from which broad categories were identified. This open-coding methodology allowed 
complete examination of thoughts and helped focus the analysis (Creswell, 2002). Throughout the analysis process, 
the research questions were revisited and the data was reviewed for convergence and divergence. The research 
participants performed member checks on the transcripts. 

Results 

Cross-case analysis 

The following underlying questions were posed by the study. What pedagogical philosophy guides 
American college faculty? Do college faculty believe in the traditional instructional model of whole class structured 
explanation, guided practice, and a common curriculum contained in textbooks? By comparison, do college faculty 
believe that a learning process which stresses student interest, initiative, and "sense-making" is more important than 
a particular curriculum and see themselves more as a facilitator and resource provider, than a source of content 
knowledge for students? The results revealed that fewer teachers professed to believe in the traditional, fixed- 
curriculum, fact, and skill -oriented model of teaching and instead the majority of the faculty members supported a 
more constructivist teaching philosophy. 

Traditionally, teaching practice has been characterized by an emphasis on skill and knowledge transmission 
from teacher to students. By contrast, constructivist theorists argue that understanding is derived from a person's 
effortful activity to integrate newly communicated claims and ideas with his own prior beliefs and understandings. 
To examine this argument empirically, the survey incorporated a set of questions designed to measure their 
philosophical preference between transmission- oriented teaching and constructivist-compatible teaching. Table 1 
represents the alternative statements of teaching philosophy (Becker, 2001). 
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Table 1 Transmission - Constructivist Philosophy Continuum 

Transmission Philosophy Constructivist Philosophy 

Continuum 



Explainer 

Students really won't learn the subject unless you go 
over the material in a structured way. It's my job to 
explain, to show the students how to do the work, 
and to assign specific projects. 

Curriculum Content 

The most important part of instruction is the content 
of the curriculum. The content is what students 
need to know and be able to do. 

Curriculum Content 

While student motivation is certainly useful, it 
should not drive what students study. It is more 
important that students learn history, math, and 
language skills in their textbooks. 

Whole Class Activities 

It's more practical to give the whole class the same 
assignment, one that has clear directions, and one 
that can be done in short intervals that match 
student's attention spans and the class schedule. 



Facilitator 

I mainly see my role as a facilitator. I try to 
provide opportunities and resources for my 
students to discover or construct concepts for 
themselves. 

Sense-Making 

The most important part of instruction is that it 
encourages 'sense-making' or thinking among 
students. Content is secondary. 

Interest Effort 

It is critical for students to become interested 
in doing academic work-interest and effort are 
more important than the particular subject 
matter they are working on. 

Multiple Activities 

It is a good idea to have all sorts of activities 
going on in the classroom. It's hard to 
organize, but the successes are so much more 
important than the failures. 



Table 2 provides information on about how much teachers’ beliefs were aligned with a set of two 
statements about teaching philosophies. There were seven possible responses for each item. The series of paired 
statements was preceded by the following statement: 

Different teachers have described very different teaching philosophies to researchers. For each of the 
following pairs of statements, check the box that best shows how closely your beliefs are to each of the 
statements in a given pair. The closer your beliefs to a particular statement, the closer the box you check. 
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Table 2. Transmission vs. Constructivist (n=5) 



Transmission Philosophy Constructivist Philosophy 





Strongly 

Agree 


Moderately 

Agree 


Agree 


Neutral 


Agree 


Moderately 

Agree 


Strongly 

Agree 


Explainer/Facilitator 








1 


2 


2 




Content/Sense-Making 










2 


3 




Content/Interest 




1 


3 


1 








Whole class activities/ 
Multiple activities 






1 


1 


1 


1 


1 



Overall, the five case study teacher responses were predominately in accordance with the constructivist 
philosophy and these views were similarly reflected in the interviews. Teacher 2 was very supportive of the 
constructivist viewpoint (facilitator) by noting “I have always perceived my role as a teacher to be that of a “tour 
guide,” leading the students on a learning journey in the subject/courses. Technology allows us to have a more 
interesting and enriching “tour” experience.” Additionally, Teacher 1 noted, “learning should be less teacher- 
centered and more student -focused.” The only variance in the constructivist approach was the value of particular 
content covered in textbooks over student interest and effort in academic work. 

Faculty type and technology usage 

Teachers 1, 4 and 5 are first-wave or self-starter types. For example. Teacher 1 noted, “I am intrinsically 
driven to use technology” and “technology works very, very well because it lets students do independent 
investigation, so that they can discover things on their own, without having to be told that a particular thing is a 
particular way.” Teacher 4 stated, “I just kinda have an affinity for that environment [technology]” 
and 

It’s kinda like speed reading a book I try to look at what’s there and how it works and then get a little 
deeper into the parts of it that I’d be more interested in and then try to figure out what there is that I don’t 
know how to do the nuts and bolts of it. 

Teacher 5 related that 

I had my first full-time regular teaching appointment as a visiting assistant professor and it [technology] 
was available and ... I quickly became someone who was a heavy user of this technology and would, you 
know, go to the IT department a lot and say I need this, I need that, can you do this for me. 

Teacher 3 represents a second-wave or traditionalist faculty type as reflected in the statements that follow 

(a) it [technology] has to be scientific, valid, reliable data to support my doing it, and (b) unless someone 
provides me with some training or ideas how, it’s just a trendy thing to do, and (c) I recently attended a 
workshop at unnamed university where they talked extensively in sharing materials, books, but no real 
discussion of technology was conducted — nobody seemed to be discussing or enamored with technology as 
part of their teaching processes 

Teacher 2 represents a third-wave or careerist faculty type as shown by the following statement 

Technology provides me with survival and continued growth as a professional educator. I see Computer 
Assisted Instruction as retirement vehicle for me personally. This delivery system for higher education is 
not a FAD, but in reality the future. We probably will always have brick and mortar course delivery 
systems, but the online feature is growing significantly. Institutions of higher education, who do not 
embrace the CAI, at least as a means of augmenting their on ground practices, will find themselves left in 
the “dust,” so to speak. 

Teaching strategies 

The second set of questions used for analysis concerns teaching strategies because the way that a faculty 
member uses computers indicates his or her underlying pedagogical philosophy. Grasha (2000) believes that current 
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and future users of technology should be concerned about instructional method bias that is “a tendency to select 
teaching processes because the structural features associated with them are personally attractive” (p. 2). 

The case study participants believe themselves to be effective and to possess the competency to influence 
student learning. Their varying beliefs about faculty roles were demonstrated in their customary classroom practice 
using technology. Faculty reported to have students build understanding through using hands-on activities. The 
more constructivist teachers described ways that technology use supports their beliefs. For example. Teacher 1 
stated, “. . . when it [technology] is used effectively, students can be actively involved and manage their own learning. 
They can discover for themselves.” Teacher 4 believes “the use of technology allows more “active learning cycles” 
for students.” Teacher 5 stated they “used technology to appeal to different learning styles and to make concrete the 
concepts.” Finally, Teacher 2 incorporates technology in teaching through the “use of multi- media approach to 
teaching, from film, overheads, videos, and now - the internet, with just-in-time use of company web-sites to 
reinforce content and bring current relevance to the discussion.” 

Major influences on teachers’ beliefs about teaching and technology 

The major factors mentioned by the teachers were pressure from colleagues, students, and outside sources. 
Teacher 3 noted, “I have a constant reinforcement by a colleague of mine who harasses me to use technology and 
also because now that I do it I find it fun” and 

I decided that in order to be in the 21st century as a professor and since I was telling my students they have 
to use PowerPoint’s I figured I’d use it myself. . .PowerPoint’s not new and interesting anymore to students 
and you know since the students are of a generation of computers and etc you gotta make things more 
entertaining for them and I don’t think PowerPoint as I am using it is particularly entertaining. 

Teacher 5 noted that the administration put pressure on faculty to integrate technology, by stating that, “I mean it 
[technology] was an institutional expectation that faculty would come to use technology.” Teacher 5 also discussed 
how easily textbook publishers are catering to technology usage through the development of PowerPoint 
presentations. 

. . .faculty becoming extensions of computer programs. . .the publishers are putting out a whole package this 
is how you teach the course and all you have to do is stand there and push the buttons and ask the questions 
that are pre-programmed into the PowerPoint's. 

Discussion and Recommendations 

This study provides insight into how a diverse number of teachers have begun to use instructional 
technologies in their instruction, and examines the relationship between personal beliefs about integrating 
technology and teaching styles and strategies. The case study interviews indicate that faculty views of learning and 
teaching noted in the surveys and interviews are reflected in the ways they use technology in the classroom. Views 
of learning and goals for instructional technology use, varied among the five teachers in the case studies, ranging 
from constructivist views for teaching and instructional technology use, to a neutral perspective. All faculty stated 
that they have used technology to support their teaching in ways they felt are appropriate. Each teacher offered 
several examples of how they use technology in a way that meshes with their teaching philosophy. The preferred 
teaching strategies and styles of teachers determined or shaped patterns of technology usage. Fewer faculty 
professed to believe in the traditional, fixed-curriculum, fact and skill-oriented model of teaching and instead 
supported a more constructivist teaching philosophy. This finding is incongruent with current literature. According 
to Bennett & Bennett, (2002) most faculty members conduct a teaching-centered classroom instead of learner- 
centered classroom. Examining the institution, where the study was performed may account for this incongruence. 
Liberal Arts educators seek “to empower our students to become lifelong learners and creative and critical citizens 
of a media -intensive world” (Scott, Chenette, & Swartz, 2002), thus a liberal arts education is more likely to consist 
of a learner-centered classroom. Also, it is interesting to note that first-wave faculty held constructivist views for 
teaching and instructional technology use while second-wave and third-wave faculty held to a more neutral 
perspective. 

Before an institution can begin to support faculty-integrating technology into instruction, the institution 
must first determine their mix of faculty types, first-wave (self-starters), second-wave (traditionalists), third-wave 
(careerists), and fourth-wave (reluctants), because the strategy needed to support technology integration into 
instruction will differ among types of faculty. 

For example, why should faculty integrate technology into their teaching if they cannot use it daily? 

Getting excited about using a new technique does not translate into being able to use it effectively. In order to 
increase the use of technology in the college classroom, college administrations need to make the technology readily 
available before asking faculty to learn and adopt it. College administrations need to develop a process by which 



161 




first-wave faculty would be empowered to integrate technology through grants (to buy more hardware and software) 
and release time arrangements. 

Second-wave faculty need a series of staff development workshops to promote the integration of 
technology into the college classroom This series of workshops would begin with learning how to use technology 
to teach (specifically how each discipline can use the technology) and proceed to learn how to use the basic tools of 
technology, and progress to intermediate workshops on integrating technology into instruction. This series would 
run continuously, with more than one level being offered simultaneously. Release-time arrangements and stipends 
might be provided to stimulate faculty participation. At the same time, to be effective in promoting the use of 
technology the college administration would need to provide the basic hardware and software for second-wave 
faculty in every office and classroom. At this point, first-wave faculty would be able to piggyback their applications 
and uses onto the available technology. 

Successful teaching requires that teachers be able to address learners’ needs and understand the variations 
in learners’ styles and approaches. Teachers can accomplish these requirements by utilizing a variety of teaching 
strategies and teaching styles. The teachers’ teaching philosophy plays a role in determining how technology will be 
integrated into instruction and understanding current instructional technology uses and beliefs of faculty provides a 
plan of action that supports effective responses. Rather than viewing technology as merely a tool for delivery, 
higher education should view technology as a means to improve learning (Wilson, 2003). Future studies are needed 
to continue developing a knowledge base about the interaction between personal beliefs of using technology and 
teaching strategies, the relationship of technology to teaching styles and the implications for the teaching styles 
adopted by faculty members. 
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Appendix A 








Under 25 



25 to 34 



35 to 44 



45 to 54 



55 to 64 



65+ 



What is your gender? 



? Female 



? Male 



3. | What is your academic rank? 



? Instructor 



? Assistant Professor 



? Associate Professor 



? Professor 



? Other 



At which higher education institution are you a member of an academic staff? 



5. Which department/school do you hold your appointment? 



How many years have you taught at your current higher education institution? 



Less than 1 year 



? 1-2 years 



? 3-5 years 



6-10 years 



11-15 years 



? | More than 15 years 



How many years have you taught throughout your higher education career? 



Less than 1 year 



1-2 years 



? 3-5 years 



? 6-10 years 



? 11-15 years 



? | More than 15 years 



I 



What is the average number of students that you teach in one term? 



? 10-20 



21-30 



? 31-50 



? 51-100 
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8. | What is the average number of students that you teach in one term? 



? 101-150 



? More than 150 



How often do you use the following technologies? 




Email at school? 



Email at home? 



A listserv? 



WWW resources at school? 



WWW resources at home? 



Other Internet resources 
(telnet, ftp, etc)? 



A word processor? 



A spreadsheet program? 



A database program? 



Presentation software? 



A web page editor? 



Distance learning? 



Desktop publishing? 



An authoring package? 



Computer-based instruction? 



A digital camera? 



A scanner? 



Video conferencing? 



WWW chat rooms or bulletin 
boards? 



WebCT, Blackboard or other 
instructional tool? 



Not 

available to 
me 


Available 
but never 
use 


Available 
but rarely 
use 


Use 

monthly 


Use 

weekly 


Use 

daily 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 


? 



10. Different teachers have described very different teaching philosophies to researchers. For each of the 
following pairs of statements, check the box that best shows how closely your beliefs are to each of the 
statements in a given pair. The closer your beliefs to a particular statement, the closer the box you check. 

”1 mainly see my role as a facilitator. I try to ''Students really won't learn the subject unless you 

provide opportunities and resources for my students go over the material in a structured way. It's my job 
to discover or construct concepts for themselves." to explain, to show the students how to do the work. 



and to assign specific projects." 



□ 


? 


? 


? 


? 


? 


? 


? 


"The most important part of instruction is the 
content of the curriculum. The content is what 
students need to know and be able to do." 


"The most important part of instruction is that it 
encourages 'sense-making' or thinking among 
students. Content is secondary.” 


□ 


? 


9 


9 


? 


? 


? 


? 


"Students must learn basic skills before they can 
master complex content." 


"Students can learn basic skills in the context of 
mastering complex content." 


□ 


9 


9 


? 


? 


? 


? 


? 



"It is critical for students to become interested in "While student motivation is certainly useful, it 

doing academic work- interest and effort are more should not drive what students study. It is more 
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important than the particular subject matter they are important that students learn history, math, and 
working on.” language skills in their textbooks." 

I I ? I ? I ? I ? I ? I ? I 



"It's more practical to give the whole class the same 
assignment, one that has clear directions, and one 
that can be done in short intervals that match 



"It is a good idea to have all sorts of activities going 
on in the classroom. It's hard to organize, but the 
successes are so much more important than the 



student's attention spans and the class schedule." failures.' 

9 I 9 I 9 I 9 I 9 I 9 I 9 



Thank you for participating in this survey. 



Appendix B 

Technology and Teaching Style Interview Guide 

Teaching Style 

1. What is your philosophy of teaching? 

2. Describe your teaching style. 

3. How does technology fit into your philosophy of teaching? 

4. How does technology fit into your style of teaching? 

Methods for Using and Integrating Technology in Teaching 

1. What initially prompted you to use technology as a teaching strategy? 

2. How did you learn to apply technology in your teaching? 

3. Describe ways in which you have integrated technology into your teaching. 

4. How does integrating technology change your role as the teacher? 

5. What specific changes to your teaching resulted from integrating technology in your teaching? 

6. What is the most compelling reason for using technology in the classroom? 

a. For not using technology? 

7. What motivates you to integrate technology into teaching? 

a. Please give a specific example 

8. What incentives drive you to integrate technology into teaching? 

a. Please give a specific example 
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Adopting an Electronic Portfolio System: Key Considerations for Decision 

Makers 

Rebecca L. Fiedler 
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The use of electronic portfolios for authentic student assessment is growing rapidly (Batson, 2002). 

Creating portfolios electronically offers a number of benefits not available using traditional paper-based portfolios. 
Advantages include the portability from one application or institution to another, wider accessibility of the portfolio, 
and the reusability of artifacts in different contexts and to create multiple portfolios. 

The two broad approaches to eportfolio implementation have advantages and disadvantages. Some experts 
advocate using generic tools (e.g., word processors, HTML editors, portable document format) to create eportfolios. 
Advantages to this approach include flexibility for portfolio authors to customize their portfolio to reflect their 
individuality and portability from one system to another. Start-up cost may also be low using this approach. 
However, generic tools carry disadvantages. Authors unskilled with the tools may suffer cognitive overload, 
producing lower quality portfolio content as they struggle to use the tools. Second, it is hard to aggregate data from 
portfolios created using generic tools. Finally, securing the contents of the electronic portfolios may be difficult if 
they are to be accessible from Internet-connected computers. 

Alternatives to the traditional generic tools are specialized electronic portfolio systems. These take 
advantage of computer databases, servers, interfaces, and custom programming. Advantages of such systems include 
more powerful data aggregation to satisfy accountability mandates, simplified security procedures, increased 
opportunities to create multiple portfolios for diverse purposes, computer- mediated communication between 
portfolio authors and their advisers, and a less steep learning curve for the portfolio author. However, disadvantages 
are evident. The portfolio author’s ability for self-expression is limited. Some customized systems are expensive to 
implement and maintain. Proprietary structures and file formats may limit portfolio portability. 

Many customized eportfolio systems can be integrated with existing student information systems and 
course management systems. An advantage of tightly integrated systems is the ability to repurpose existing 
information and artifacts for use in portfolios. However, tightly coupled systems such as these require security to 
prevent unauthorized access to private personal data. 

Selecting an eportfolio system requires rigorous attention to pedagogical and technological considerations. 
Pedagogical considerations include the level of users' technical skills, user access to technology, staff development 
needs, curricular standards to be met, the level of creativity desired, access to student reflections, and data 
aggregation needs. Additional pedagogical concerns are illustrated in the top half of Figure 1. 

Technological considerations are equally complex, including budgets for hardware, software, and technical 
support, the capacity of the institution’s infrastructure, system scalability, usability issues including plug-ins and 
utilities, authentication systems, required technical standards (e.g., ODBC- and SCORM compliance) and ongoing 
system maintenance. Additional technological concerns are illustrated in the bottom half of Figure 1. 

As technological and pedagogical considerations are addressed, a set of requirements will emerge. The 
requirements and preference decisions, along with the results of usability testing, should guide selection of an 
electronic portfolio system. This paper addresses many of these decisions in detail, describes selected electronic 
portfolio systems, and identifies additional eportfolio resources. 

Electronic Portfolios: An Overview 

For many years, artists have used portfolios to showcase their work, demonstrate their skills, and find 
employment. Portfolios are now being used in a variety of fields, including, but not limited to the humanities (Reiss, 
2001), information literacy (Fourie & van Niekerk, 1999, 2001), management (Zalatan, 2001), and nursing (Lettus, 
Moessner, & Dooley, 2001). Portfolio assessment is also gaining popularity in K-12 and higher education as both 
teachers and students collect artifacts to demonstrate skills in specific areas (e.g., Shulman, 1998; Carpenter, Ray, & 
Bloom, 1995; Karoly, 1996; Lyons, 1998; Snyder, Lippincott, & Bower, 1998; Quinlan, 2002). Several definitions 
of portfolios have been offered. One of the most frequently cited is that of Paulson, Paulson, and Meyer (1991): “A 
portfolio is a purposeful collection of student work that exhibits the student’s efforts, progress, and achievements in 
one or more areas. The collection must include student participation in selecting contents... and evidence of student 
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self-reflection’' (p. 60). Similarly, Shulman (1998) of Stanford University defines a teaching portfolio as “the 
structured, documentary history of a set of coached or mentored acts of teaching, substantiated by samples of student 
portfolios, and fully realized only through reflective writing, deliberation, and conversation" (p. 37). Barrett (2001b) 
believes “an electronic portfolio is really a living history of lifelong learning.” 

This paper discusses a cross-section of issues related to portfolios, eportfolios, and eportfolio development 
and implementation. Topics include a look at the dilemmas faced by implementers, technological and pedagogical 
concerns raised in the literature, information about the portfolio development process, clarification of important 
terms, criteria for evaluating eportfolio tools, and an examination of various eportfolio tools. Important concerns, 
both technological and pedagogical are addressed and a structure for thinking about those issues is proposed. 

Finally, the paper closes with a list of useful resources and organizations for the decision-maker interested in 
locating additional information about electronic portfolios. 

Defining Electronic Portfolios 

As technology becomes increasingly available, student portfolios are moving from paper formats to 
electronic formats. Experts distinguish between various electronic portfolios and electronic portfolio 
implementations in subtly different ways. Barrett (2000) defines electronic portfolios to include “the use of 
electronic technologies that allow the developer to collect and organize artifacts in many formats (audio, video, 
graphics, and text)" (p. 15). Barrett emphasizes that “an electronic portfolio is not a haphazard collection of artifacts 
(i.e., a digital scrapbook or multimedia presentation) but rather a reflective tool that demonstrates growth over time" 
(p. 15). Barrett (2003) distinguishes between electronic portfolios and online assessment management systems. 

Many teacher education systems are implementing electronic portfolios to meet NCATE standards. These systems 
often resemble an online assessment management system, complete with numerical scoring, rubrics, and statistical 
analyses. Trent Batson (2002) uses the term eportfolio to refer to “database-driven, dynamic web sites, not static, 
HTML-driven sites" (p. 2). The online assessment management systems are typically database-driven. These 
distinctions can result in vastly different implementations. 

The use of eportfolios is growing in popularity. Electronic portfolios are used for such diverse purposes as 
assessment, accreditation, reflection, and professional development. An increasing number of eportfolio developers 
are entering the market, facilitating the portability of file types including text, graphics, video, audio, photos, and 
animations. This increasing popularity is due, at least in part, to the increasing use of digital formats for student 
work and the need to organize, search, document, and transport this work. These activities take advantage of 
computer capabilities of storage, display, retrieval, and communication to change curricula (Batson, 2002). 

The American Association for Higher Education (2003) has proposed a “Taxonomy for Electronic Portfolios” to 
facilitate discussions about the various kinds of eportfolio implementations. This taxonomy has three main 
discriminators: context, author, and purpose. Understanding these dimensions for your specific need is important to 
select the “right" portfolio system or tool. The context describes the setting in which the portfolio was developed. 
Contexts include courses, programs, institution, inter-institutional, and independent portfolios. For example, 
depending on the context, data aggregation may or may not be an important factor to consider. The author of an 
electronic portfolio may be a student, faculty member, administrator, organization, or other individual including 
those not specifically affiliated with the learning organization. Knowing the portfolio authors, and their skills, 
should influence your decision. Purposes for portfolios include development (self-assessment, advising, 
documenting learning over time, documenting professional development, building the curriculum, adding to the 
knowledge base of or among the disciplines), evaluation (achievement of learning outcomes, high stakes evaluation, 
accreditation, promotion and tenure), and presentation (showcasing achievement, publicizing organizational 
reflection and progress, and responsiveness to state and national need for information). The variety of purposes will 
determine the level of creativity and flexibility required of the selected system. 

Dilemmas Eportfolio Authors Encounter 

Electronic portfolios can be developed using one of two broad approaches: using generic tools (word 
processors, HTML editors, portable document format, authoring software) or using customized information systems 
including databases, servers, interfaces, and custom programming (Gibson & Barrett, 2002). Each approach has both 
advantages and disadvantages, forcing eportfolio authors to make choices that are oftentimes difficult. 

Joanne Carney (2002) completed case study research that examined the decision-making of electronic 
portfolio authors. Her findings help illuminate the decision about which type of electronic portfolio tool to 
implement. She identified six dilemmas that must be confronted by those interested in electronic portfolio 
implementation. Those dilemmas are: 
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Multiple-purpose dilemma - Students are encouraged to create portfolios for multiple purposes and 
multiple audiences. Too often, these purposes are in conflict with each other and the result is that none of the 
purposes are served well. An example of this type of conflict can be found in pre -service teacher education. Pre- 
service teachers are asked to include reflections on their teaching internship in their portfolio. They are encouraged 
to reflect on their mistakes and share what they learned from that mistake. However, teacher educators also suggest 
the portfolio be used for credentialing purposes. Students may be understandably reluctant to share the same 
information about their mistakes with a licensing agency that they share with a trusted professor. This conception of 
the portfolio also being used in a high-stakes assessment situation often motivates the portfolio author to downplay 
any problems they had in teaching despite the learning benefit from a critical examination of such a problem. 
Portfolio authors are likely to experience a constant tension between the conflicting purposes of their portfolio 

Personal -rev elation dilemma - Electronic portfolios are frequently placed on the Internet for a worldwide 
audience. Revealing personal difficulties is a source of discomfort for many people and can be used against them in 
countless ways. 

Cognitive-overload dilemma - This dilemma is exacerbated by the use of generic tools. Portfolio authors 
lacking the technological skills to produce an electronic portfolio spend cognitive capital coping with their 
technological deficiencies at the expense of portfolio content. Even portfolio authors with a high degree of skill 
spend capital planning and executing specific tasks and exploring new capabilities. With customized systems, the 
cognitive overhead is less than that required to use generic tools. 

Self-expression dilemma - Electronic portfolio authors using customized systems are often constrained by 
the systems they use. These authors exert less control over format, appearance, and structure than a portfolio author 
skilled in using generic tools. Portfolios are often considered a very personal reflection of self, and this constraint 
can be a source of dissatisfaction. On the other hand, the unskilled users of this system can readily select an 
aesthetically -pleasing, but pre -determined template to create an eportfolio. 

Dead-end dilemma - One of the original purposes of portfolios was to serve as a map or plan for future 
growth and development. Authors of paper portfolios often consider their portfolio to be “done” while electronic 
portfolio authors are more likely to consider their portfolios “a work in progress.” Authors using generic tools have 
portable files that can be taken from school to school or into the workplace. Customized systems using proprietary 
file formats may cause users to hit a dead-end when they attempt to move their files or teacher candidates to 
abandon their portfolio upon graduation or the next bill to renew the service. 

Data-aggregation dilemma - Customized portfolio systems are specifically designed to aggregate data for 
such purposes as program evaluation and accreditation. With that as a goal, institutional requirements may 
discourage portfolio authors from documenting deficiencies, despite the potential learning opportunities from a 
closer examination of such difficulties. 

Advantages and Challenges in ePortfolio Implementation 

The use of electronic portfolios offers a number of advantages over traditional paper-based portfolios, such 
as portability, accessibility, distribution ability, and repeatability of performances (Sheingold, cited in Barrett, 1998). 
With portfolios, students participate in their own assessment, rather than rely solely on a teacher’s evaluation of their 
work(Mullin, 1998). 

The ePortConsortium White Paper (2003) contends that the rise in e-learning has contributed substantially 
to the viability of electronic portfolio adoption. At institutions where elearning is in place, there is an existing 
network that can also be used for an electronic portfolio initiative. Many elearning systems use database-driven 
course management systems. Oftentimes, these systems capture information that can be repurposed for an electronic 
portfolio program. Furthermore, many of the artifacts a portfolio author may decide to use are already available in 
electronic formats and easily transferred to an eportfolio system. Additionally, the data aggregation capabilities 
already present in elearning databases and electronic portfolio systems can facilitate working with accountability 
mandates existing in education today. 

Well-designed electronic portfolio systems facilitate ongoing dialogue between portfolio authors and their 
audience members. Annotation capabilities and threaded discussions allow the preservation of such dialogues for 
documentation of learning and evidence included in reflections. Electronic portfolios take advantage of hyperlinking 
to provide alternative routes through an individual portfolio. Furthermore, authentication and security practices 
allow an eportfolio author to determine who is able to see specific artifacts and who is denied this access. 

Finally, electronic portfolios offer opportunities in the area of career development. Employees and 
eportfolio authors can maintain and update their personal information to showcase skills, certifications, 
accomplishments, and awards (ePortConsortium, 2003). However, issues of verification must be addressed (Batson, 
2002). There are foreseeable circumstances in which a portfolio reviewer, be it a potential employer or graduate 
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school admissions counselor, would want to ensure work represented in the portfolio was actually that of the 
portfolio author. 

Despite the numerous benefits afforded by electronic portfolios, eportfolio innovators must address a 
number of challenges. The authors of the ePortConsortium White Paper (2003) highlight seven areas in which 
challenges may arise: hardware and software, security and privacy, intellectual property and digital rights, usability, 
assessment, acceptance and managing expectations, and long-term maintenance. 

1. Hardware and software challenges include the decision to use a generic tool or customized portfolio 
system as well as identifying funding for long-term licensing of such products. Further considerations include 
interoperability with existing systems such as student information systems and human resources system. Systems 
should be scalable to support further growth of the institution. Levels of expertise residing in the organization must 
be considered to determine development and deployment options. 

2. Security and privacy issues must be addressed to determine who will access portfolio content and system 
data. Policies to address such concerns must be specified before extending access to external audiences such as 
potential employers and program evaluators. The institution’s responsibilities under the Federal Educational Rights 
and Privacy Act (FERPA) must also be considered. 

3. Intellectual property and digital rights concerns are growing every day. Issues of ownership of materials 
uploaded to electronic portfolios must be addressed and methods to resolve disputes put in place. Mechanisms for 
verifying and preserving records of authorship should be implemented. “Fair use” guidelines must be made clear. 

4. Usability issues to address include identifying the plug-ins and utilities required to use the eportfolio 
system, the file formats that will be supported, browsers and computer operating systems that can be used, and the 
bandwidth and other technologies required to create and maintain an electronic portfolio. Additionally, the structure, 
navigation, and system performance of the eportfolio system will enhance or detract from usability. 

5. Assessment challenges include decisions on individual, course, program, and institutional assessment. 
Curricular and performance standards appropriate for such assessments must be identified and decisions about how 
assessment data is maintained must be made. Finally, decisions about who will have access to assessment data must 
be considered. 

6. Acceptance and managing expectations is a challenge with any large-scale implementation. Managing 
user expectations and fostering acceptance among users are critical tasks to ensure long-term success. Clarifying 
expectations and responding to suggestions from a diverse user community including alumni, staff, students, 
evaluators, and administrators must be addressed. 

7. Long-term maintenance issues include an author’s ongoing access to his or her eportfolios. Decisions 
about how and when to delete files or move them to long-term storage must be addressed. Garbage collections 
routines to delete orphan pages and dead links should be examined. 

Important Concerns to Address 

Implemented of electronic portfolio programs must address a number of issues, including both 
technological and pedagogical concerns. 

Technological Concerns 

Batson (2002) has identified four areas of concern: storage, security, certification, and industry stability. 

1. Storage - An organization implementing an eportfolio program must consider how work will be stored 
and for how long. Furthermore, multimedia and graphics files can be quite large, requiring large quantities of costly 
storage space. In addition, accessing older storage media is problematic. For example, machines capable of reading 
data stored on a 5.25” disk are rare today. 

2. Security - Maintaining the security of personal information is an important consideration among 
information technologists today. Electronic portfolios may be containers for private information including social 
security numbers, grades, and intellectual property. A security breach could potentially expose the user to identity 
theft and an organization to a lawsuit. 

3. Certification - If an educational organization intends to certify work contained in electronic portfolios is 
authentic, a system needs to be created enabling faculty to prevent further changes to certain artifacts and data. 
Authenticity is critical for effective assessment, evaluation, and credentialing. 

4. Industry stability - Many companies offering electronic portfolio programs are relatively new and often 
small. Educational institutions making a long-term commitment to an electronic portfolio program will face tough 
decisions relying on new small vendors with an uncertain future. 

The ePortConsortium’ s White Paper (2003) addressed additional concerns including system infrastructure, 
interoperability, and technological standards. System infrastructure concerns examine the relationships between 



170 




eportfolio systems and other campus systems already in place. A single department eportfolio system, such as 
University of Central Florida is undertaking Fall 2004, can be a standalone implementation with little to no impact 
on existing system infrastructure. While integration of an eportfolio system with a Student Information System (SIS) 
is achievable, it is not necessary. One advantage of a stand alone implementation is rapid deployment, however a 
disadvantage can be duplication of effort and content across various campus systems. Other deployment options 
include integrating the eportfolio system with a course management system or full integration with campus 
enterprise systems. An advantage to these types of deployment is a single authentication system for all campus 
systems. Specific security and certification issues can be addressed more broadly in this type of implementation. 
Additionally, such systems may be tightly integrated with course management systems, learning systems, grants 
management systems, and other campus wide systems. 

Interoperability and technological standards considerations can insure eportfolio systems are able to 
communicate with other campus systems to receive and pass information needed by the various systems. 

Information requests may include access to data created by and about users, standardized data structures, and 
verification and digital rights management. Interoperability will also assist in managing workflow between various 
constituents. Adoption of technological standards can allow content to be reused, integrate data from one system 
into other systems, and facilitate authentication and authorization. 

Usability and accessibility by “special needs” populations must also be considered. Adherence to specific 
document standards (i.e. PDF, jpg, etc.) can help insure archived documents can be read in the future. Investigation 
into the National Archives and Records Administration’s (NARA) PDF/A standard or some other long-term archival 
format is important. Widespread adoption of carefully defined technological standards can facilitate portability of 
eportfolios and data from one institution or one system to another. This is essential for the long-term usefulness, 
accessibility, and viability of electronic portfolios systems. 

Pedagogical Concerns 

Issues of validity and reliability appear repeatedly in examinations of large-scale portfolio assessment 
deployments (Koretz, 1998; Murphy, Bergamini, & Rooney, 1997; Stecher, 1998). Recently, Wilkerson and Lang 
(2003) reported on the legal and psychometric issues surrounding the use of portfolios for high-stakes assessment 
purposes such as teacher certification assessment. They raised important concerns about the psychometric properties 
of validity, reliability, absence of bias, and fairness. 

Gearhart and Herman (1998) examined the social context of portfolio development. They found wide 
variability in how portfolios are implemented and the time teachers provide for students to receive assistance with 
revisions and choosing artifacts. Though such concerns are less critical for classroom assessments where a teacher 
understands the context in which the portfolio was constructed, this social milieu introduces reliability problems for 
large-scale assessments. Heller, Sheingold, and Myford (1998) examined portfolio raters and their reasoning while 
conducting portfolio assessment. They were specifically interested in determining when problems with the rating 
process introduced threats to validity. They noted raters found portfolios with uneven evidence were most difficult 
to rate. 

In addition to the technological concerns of storage, security, and portability, Barrett (2002) added the 
pedagogical concerns of reflection, publishing, and linking and grouping to the list issues in implementing electronic 
portfolios. She stressed the importance of reflection to provide portfolio authors an opportunity to demonstrate 
achievements and to set future learning goals and direction. Under the category of publishing, she raised issues such 
as the ability to individualize portfolio presentations to avoid “cookie cutter” effects and facilitate creativity. Further, 
she endorsed the ability to create a variety of portfolios for different purposes. Types of portfolios might include a 
learning portfolio focusing on the learner as the audience, a highly structured assessment portfolio to demonstrate 
achievement and receive faculty feedback, employment portfolios, and showcase portfolios demonstrating growth or 
highlighting specific achievements. 

Work by Wolfe and Miller (1997) indicated teacher time and perceived difficulty in scoring are the biggest 
barriers to portfolio assessment and such concerns must be carefully addressed. They also noted perceived barriers 
decreased as teachers gained more experience with portfolios. 

Kathleen Yancey (2001) wrote about the issues to be addressed in the portfolio planning process and has 
identified six critical issues, both technological and pedagogical. These included storage, leveraging the potential of 
the electronic environment, defining required technical skills for both students and faculty, determining the level of 
creativity to be supported (i.e. templates or individual creations), determining a schedule for faculty review and 
feedback, and defining the life of the portfolio. 
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Developing Portfolios 

To understand the technological and pedagogical considerations associated with developing electronic 
portfolios, interested parties can review work from the traditional paper portfolio movement. Burke, Fogarty, and 
Belgrad (1994) have elaborated on a 10-phase portfolio development process (see Figure 1). Briefly, those phases 
involve: 

• Project the purposes and uses of the portfolio to be developed. 

• Collect and organize the artifacts to include in the portfolio. A variety of organizational schemes 
and devices are appropriate. 

• Select the artifacts to be included. Selections may be based on general guidelines. 

• Interject the personality of the portfolio author by color selections, choices regarding page layout, 
or setting the mood or tone. 

• Reflect metacognitively. This involves descriptions of each artifact to help the portfolio reader 
understand the importance of the selected artifact and how that artifact fits into the overall 
portfolio and portfolio goals. 

• Inspect and self-asses goals. This is an examination of the overall portfolio to determine what, if 
any, material must be added or re -aligned. 

• Perfect, evaluate, and grade (if you must). In this stage, portfolio authors add the finishing touches 
and prepare the portfolio for the next stage. Teachers may choose to grade a portfolio at this stage. 

• Connect and conference. Considered by Burke, Fogarty, and Belgrad to be a critical stage in the 
assessment process, the portfolio conference allows portfolio authors to share their work with 
parents and receive critiques from others. 

• Inject and eject. It is in this stage that the portfolio author performs ongoing maintenance and 
updating of his or her portfolio. New materials are added and obsolete materials are removed. 

• Respect accomplishments and show with pride. In this ongoing stage, portfolio authors can show 
their portfolios to others. Such exhibitions offer insight about the portfolio author to portfolio 
viewers. 

Barrett (200 la) views “portfolios as a process rather than a product - a concrete representation of critical 
thinking, reflection used to set goals” (p. 1). She situates her discussion of developing electronic portfolios in two 
bodies of literature - that of portfolio development and of multimedia development. Barrett (2000) developed a five- 
stage model of electronic portfolio development based on these discussions. 

The first stage defining the portfolio context and goals. In this stage, the portfolio author must identify the 
purpose and audience for the portfolio, the goals or performance standards that will provide the organizing 
framework for the portfolio, and identify the software to develop the portfolio. 

The second stage is a working portfolio. In this stage, the portfolio author uses the selected software to 
collect digital artifacts representing achievements and efforts. The third stage is the reflective portfolio in which the 
learner chooses specific artifacts to be included in the portfolio and completes reflections. Care must be taken to 
protect the privacy of these reflections as appropriate. 

The fourth stage, the connected portfolio, is unique to the electronic portfolios in that it allows the portfolio 
creator to insert hyperlinks to various documents and artifacts to create additional meaning. Documents are 
hyperlinked in a way that allows navigation. All artifacts have been inserted into the document and the portfolio is 
ready to present to others. The fifth and final stage is the presentation portfolio. In this stage, the portfolio is shared 
with appropriate audiences in a medium of the learner’s choosing. Some are published on the Internet, CD, or video. 
At this stage, some advocate using student -led conferences or interviews to present the portfolio to the appropriate 
audience (Lyons, 1998; Shulman, 1998; Snyder, Lippincott, & Bower, 1998). 

Developing Eportfolio Tools 

Today’s discussions about eportfolio tools involve the technical community and the academic community. 
At times, commonly used terms have different meanings in the different communities. Usage of these terms appears 
to be imprecise and these authors believe discourse can be enhanced by a clarification of the terms. 

Standards vs. Requirements A number of organizations are currently trying to define standards and 
requirements for electronic portfolio systems. The term “requirements” is ambiguous. Members of the instructional 
community speak of requiring a certain number of artifacts, or a certain type of artifact to provide evidence of 
specific competencies. In the technical community, the term “requirements” refers to a list of things adopters want 
the system to be able to do. 
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The term “standards” also has multiple meanings. In the context of instruction, “standards” are typically 
promulgated by an external organization such as an accrediting body, a department of education, or professional 
organization. In the technical community, a “standard” means a rule or description to which other things must 
conform. Examples include units of measurement (inch, pound, or hour) or specific protocols (Ethernet or VHS). 

Such technical standards are typically owned, controlled or defined by a specific organization. For 
example, the Institute of Electrical and Electronics Engineers (IEEE) defines many of the standards associated with 
computer networks. Product manufacturers adhere to these standards to ensure Toshiba, Dell, IBM, and Apple 
computers can share data on the same network. 

Similarly, networking companies adhere to standards to allow various products to interoperate. For 
example, IEEE’s 802.11 standard specifies requirements to insure the interoperability of wireless networking. As a 
result, wireless access points and wireless network cards manufactured by a number of different companies can all 
work on the same network. The first author’s Apple AirPort card, her professor’s Cisco wireless networking card, 
and her friend’s Netgear wireless networking card all allow wireless access to the UCF wireless network. The brand 
of the university’s wireless access point is unknown and unimportant from a user perspective. The products all work 
together (a user requirement) due to meeting certain specifications (a standard). 

Open Source and Open Standards Open source is increasingly visible in today’s higher education 
institution. The open source software movement has provided successful software projects including Linux, Apache, 
OpenOffice, and Sakai. The open source software movement is based on the wide distribution of source code, 
enabling any programmer to make improvements and modifications for further distribution. Such software is 
typically provided at no cost, or for nominal fees. However, this is not always how the term “open source” is used. 
Some proprietary vendors make their source code available to customers for increased customization and improved 
interoperability. It is still considered by many to be open source software because the source code is open. 

Open standards, such as Shareable Content Object Reference Model (SCORM) are generally driven by 
organizations attempting to define protocols to increase interoperability of systems. Such organizations include IMS 
Global Learning Consortium, Advanced Distributed Learning (ADL) network, and the IEEE Learning Technology 
Standards Committee. 

Over a period of six years, the University of Minnesota developed an electronic portfolio system for use by 
students, faculty, and staff. Recently, the code for this system was donated to the Open Source Portfolio Initiative 
(OSPI), a collaborative effort to create an electronic portfolio system by the open source software community 
(Treuer & Jenson, 2003). According to the Open Source Portfolio v. 2 Concept Paper (OSPI, 2003), efforts to 
further develop this code place interoperability and adherence to industry standards as a high priority. Similarly, the 
ePortConsortium (2003) advocates for adherence to open standards. 

As electronic portfolio adopters begin to consider the eportfolio tools they will use, a decision about open 
source and open standards is necessary. James Dalziel (2003) examines these two movements and the interactions 
between them. The decision is not an easy one, involving tradeoffs that must be considered carefully. 

Evaluating ePortfolio Tools 

The discussion in this paper thus far has focused on describing and defining terms associated with 
electronic portfolios. With the foundation for decision-making laid, the remainder of the paper addresses how an 
eportfolio implementer can evaluate and select eportfolio tools. 

Eportfolio adopters must carefully examine their context and requirements to develop criteria against which 
to evaluate eportfolio systems. Gibson and Barrett (2002) offer 10 broad criteria to consider when comparing the 
generic tools approach to the customized system approach. Those 10 criteria are: 

1. Planning and goal setting - A portfolio can be used as a tool for portfolio authors and advisers to set 
future goals and plan further learning experiences. In a customized system, this process may be prompted by the 
system and facilitated by documentation of conversations and linking to potential goals and standards. 

2. Framework for creativity - A portfolio reflects the creativity and personality of the author. 

Considerations under this category include the electronic portfolio author’s ability to individualize the portfolio 
using the tools at hand. A portfolio author using generic tools is only limited by the author’s skill with the tool. 
Customized systems may offer few, if any, choices for structure and appearance. 

3. Communications - Planning, developing, and publishing an electronic portfolio requires communication 
between the portfolio author and a mentor or adviser. An electronic portfolio system should be evaluated for 
capabilities to support this ongoing dialog. 

4. Collaboration tools - Examples of collaborative tools include threaded discussions and whiteboards. 
Electronic portfolio systems support collaboration to varying degrees. 
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5. Reflective process - An electronic portfolio should support learner reflections. Multiple audiences need 
to be accommodated which will likely involve addressing security concerns. 

6. Connection capabilities - An electronic portfolio must have the capability to link various work products 
and reflections. The system should support linkages for multiple purposes and multiple audiences. 

7. Organizational flexibility - Electronic portfolios should be organized in flexible ways to accommodate 
multiple views, purposes, and audiences. 

8. Display flexibility and transportability - Portfolio authors should be able to display their electronic 
portfolios in various ways. Proprietary display systems will limit this flexibility. 

9. Data and information - Data and information may be aggregated to support program evaluation, 
accreditation, and curricular decision-making. Data collection efforts must consider privacy and confidentiality 
concerns of electronic portfolio authors. 

10. Start-up costs and maintenance - Start-up costs and long-term maintenance needs must be calculated to 
determine if the portfolio system is cost-effective. 

Additionally, eportfolio adopters will wish to review a variety of requirements and standards proposed by 
various groups. Adherence to certain technical standards may form the basis of some requirements against which an 
electronic portfolio system must be evaluated prior to adoption. Lists of such standards can be found in Treuer and 
Jenson (2003), as well as the ePortConsortium White Paper (2003). 

Consumer’s Guide to Selected Electronic Portfolio Systems 

To analyze four e-Portfolio systems (or assessment management systems), from a consumer’s perspective, 
the authors used the system based on several assumptions about the institution environment and institutional goals, 
the system users, and the targeted audiences. The eportfolio systems selected for further investigation were LiveText, 
TaskStream , The OSPI, and Chalk and Wire. 

Assumptions about institutional environment and goals 

Implementation of the eportfolio system would take place in a higher education institution with at least 
bachelor and masters programs in multiple disciplines. The authors assumed the organization required an electronic 
portfolio system capable of integrating with existing student and course management systems, hardware, and 
software, as well as meeting the diverse present and future needs of faculty, students, and the community. However, 
the authors did not review interoperability with other systems. 

The authors assumed the institution provided on-campus computer access and a computer network to 
faculty and students. The computer hardware specifications assumed at least 256 megabytes of RAM, a 20 gigabyte 
hard drive, a microphone and speakers, a scanner, a read/write CD or DVD drive, a floppy drive, a high-speed USB 
port, and Internet access. Installed computer software was assumed to include an office suite application (i.e., word 
processor, spreadsheet, and presentation tools), Internet browser, Adobe Acrobat reader, and other freeware 
necessary to view some Internet documents and Web sites. Minimal graphic software tools (i.e., paint, Netscape 
Communicator, etc.) were also assumed to be available. 

The institution’s document management and storage systems assumed access through Internet or network 
to a central server, either located at the institution or through the service provider, and the e-portfolio system would 
need to provide a method for organizing and maintaining digital artifacts. 

Assumptions about system users 

For this evaluation, the authors envisioned two primary audiences: instructors and students. To conduct the 
evaluation, we assumed both instructors and students were proficient in performing basic tasks in word processing 
and spreadsheet software, e-mail, and Internet browsing and therefore capable of entering data into a pre -designed 
form and uploading various document formats (e.g., doc, xls, pdf, ppt, jpg, etc.) to create an artifact database. 

From the instructor’s perspective, the purpose of the e-Portfolio is to demonstrate the student’s learning and 
growth as he/she progresses through the program. As an assessment tool, the eportfolio also provides physical 
evidence of knowledge and skills gained, as well as reflections of the learning experience and interpretation of 
learning and skill development. The instructor requires an eportfolio system providing the student flexibility to 
creatively demonstrate accomplishments, as well as development of evaluation rubrics based on program standards 
and generation of both individual and aggregated assessment data for student achievement of standards reports. 
Instructors also desire the option of developing and presenting a collection of exemplary portfolio artifacts for 
illustrative purposes. 
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Although the student may recognize the value of the instructor’s application of e-Portfolios in learning, 
his/her perspective also focuses on future employment application. The student requires an e-portfolio system 
providing flexibility and creativity to effectively market him/herself. Students need to be able to publish the 
portfolios for presentation in a variety of formats: hard copy, videotape, DVD, CD-ROM, and Internet. 

In terms of context, the authors assumed the eportfolio system would typically be used as part of classroom 
instruction. As such, professors and instructors provide criteria and standards to use when selecting artifacts to 
include in the portfolio, along with a rubric explaining how the portfolios would be evaluated. The authors assumed 
the e-portfolio system would need to have both public and private student self-reflection areas, including 
documentation of future learning goals. Likewise, a need for the system to provide for confidential feedback from 
instructors was an assumed functionality. 

Assumpti ons about portfolio audiences 

Generally speaking, the authors assumed the audience to receive a complete eportfolio would be 
individuals from a higher education institution, instructors, students, their families, and potential employers. At a 
minimum, all members of the targeted audiences had access to a computer with Internet access or a CD or DVD 
player. Software requirements included an office application suite, Adobe Reader, and a web browser. 

Although cost was a primary factor in the institution’s decision criteria, faculty and student usability was 
more critical. The criteria used to evaluate e-portfolio system usability originated from work by Gibson and Barrett 
and included: 

• Planning and goal setting: Students’ portfolios can easily be evaluated against a variety of standards 
templates. 

• Creativity ability: Students can creatively build and edit portfolios to best illustrate their individuality. 

• Facilitated communications between instructor and student: System can provide internal message 
system, as well as options for instructors and peers to comment on each other’s work. 

• Collaboration tools: System can provide collaborative learning and online collaboration opportunities 
among peer and support circles. 

• Support for reflective processes: System offe rs areas for students to communicate reflective aspects of 
projects, assignments, and courses. 

• Connection and linking capabilities: System can provide methods for linking assignments, reflections, 
and other creative work products to tailor the portfolio for different audiences. 

• Organizational flexibility: System can provide opportunities for reframing portfolios for different 
applications and situations. 

• Display flexibility and transportability: Eportfolio can transfer to other formats and platforms. 

• Data and information: System can provide a variety of customizable and formatted reports generated 
by authorized users. 

• Start-up costs and maintenance: System start-up costs and ongoing maintenance are minimal due to 
ease of system administration. System administration can provide ability to associate users and 
courses; allow users to set-up and edit course profiles; offer a robust, useful, and accurate help system, 
providing clear directions regarding system features and use; enable departments to customize the 
system for unique purposes. 

• The authors of this paper added system functionality and usability: Students can log in and out of the 
system easily; determine which system’s options are available for their use; select user profiles and 
types; easily create, organize, and upload work online; submit assignments online. Instructors can 
easily create assignments online, providing corresponding instructions and assignment details. System 
security features are available and easy to apply by both instructors and students. 
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LiveText 



Criteria 


Results 


Planning and goal setting 


Instructors have convenient access to a wide variety of standards which can be used 
to create individual or institution-wide rubrics for evaluating student portfolios. 


Creativity ability 


Limited creativity is supported with the ability to upload and incorporate images. 
Images can be placed above or below text, or to the left or right of text, depending 
on the user’s wishes. The Java editor offers user control on heading type, a limited 
font selection, styles (bold, italic, underline), a small color palette, and access to 
bulleted and numbered lists. Six site theme colors are available: sage, lilac, classic 
blue, cranberry, grape, and sunset. The selected theme is applied across the entire 
site, not just one portfolio. 


Facilitated communications 
between instructor and 
student 


Portfolios can be shared between instructors and students. The instructor receives 
portfolios for review in a section of “My Desk”. Rubrics to evaluate specific 
documents or portfolios in general are available at the institutional level or can be 
created by individual instructors. These rubrics offer commenting capabilities. 
Users can enable discussion threads on each page. 


Collaboration tools 


The user can enable discussion threads on each page. Students can share their 
portfolios and documents with instructors, peers, and visitors. Commenting is 
supported. 


Support for reflective 
processes 


Institutions can create sample reflections or instructions for reflections within 
templates. These can be modified by the user for individual use. Artifacts can be 
attached or linked to the reflections. 


Connection and linking 
capabilities 


Linking capabilities are good, supporting links to documents both internal and 
external to the system. LiveText also supports attachments. 


Organizational flexibility 


Permits creation of courses, lesson plans, portfolios, and projects; User can create 
multiple instances of each project type and link from one component to another. 
Supports multiple portfolios with user-defines sections in each. Also allows 
institutions to create templates for users to modify as needed. 


Display flexibility and 
transportability 


Offers option to print, permitting selection and preview of print job. Also exports 
files to a zipped html archive of documents, images, and attachments. LiveText 
branding is removed and users can work with CSS to apply a new look to their 
work. 


Data and information 


LiveText is strong in this area, collecting and aggregating data across professors for 
use in the accreditation process. 


Start-up costs and 
maintenance 


Start-up cost is $79 or $99 per student, depending on package selected; 
maintenance $5/month after graduation. Cost is shifted to the student. 


System functionality and 
usability 


Login and logout is easy. Availability of options is readily apparent. Uploading 
work is easy. Sharing work with others is not difficult; the default security settings 
protect individual work. The system is complex enough to be daunting to the 
novice user. Highly proficient users should be able to figure out the system fairly 
easily. On the Macintosh platform, the biggest usability problem is “bugginess” 
with at least one bug resulting in lost and irretrievable data. 



Bugs encountered: 

Loading Preview - While creating a new portfolio and choosing a template, the first author saw a message 
LiveText was loading a preview. The preview never loaded. The author attempted loading twice using IE 5.2 on 
Mac OS 10.3.5 with 640 MB of RAM. 

Java Editor - Scrolling through long documents resulted in erratic behavior, making it difficult to select 
large blocks of text (for example, to change fonts). Apostrophes and quotation marks regularly turned into question 
marks, probably due to copy and paste procedures. 
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TaskStream 



Another e-Portfolio system evaluated based upon the above criteria was TaskStream 
( http://www.taskstream.com) . The opening screen for TaskStream provides a logical, organized view of the system 
tools and features, making it extremely user friendly. 



Criteria 


Results 


Planning and goal setting 


TaskStream offers a wide variety of standards for aligning student portfolios with 
the curriculum, as well as tracking standards and generating reports. 


Creativity ability 


Due to the templates offered through TaskStream, students have limited creativity 
options for displaying and demonstrating work. 


Facilitated communications 
between instructor and 
student 


TaskStream offers an internal messaging system for direct communication 
between the instructor and students. New messages are indicated by a blinking 
“New Message” icon in the upper right corner of the screen. 


Collaboration tools 


TaskStream allows students to share their e-Portfolios with other students, both 
within the system, as well as external e-mail. Based upon the level of access 
provided, students could collaborate to create a portfolio assignment for a given 
class. 


Support for reflective 
processes 


TaskStream provides a text box for student reflection at the time the student is 
uploading the artifact. Students can upload additional documents to support their 
reflective comments. 


Connection and linking 
capabilities 


Although TaskStream offers opportunities for providing links to other online 
work, as well as uploading work, limited interaction can be created between and 
among work products. 


Organizational flexibility 


TaskStream appears to offer some flexibility for reframing portfolios for a variety 
of applications (e.g., school, job search, etc.) through the “Publish/Share” option. 


Display flexibility and 
transportability 


Student portfolios are created and maintained on the TaskStream system. 
Although the uploads and links could be transported to other e-Portfolio systems, 
students would basically be re-creating their portfolios on the new system. 


Data and information 


TaskStream offers a variety of formatted, printable reports. 


Start-up costs and 
maintenance 


TaskStream provides system administration options to both instructors and 
students. Students can associate, set-up, and edit course profiles. The help system 
is useful and comprehensive, providing easy to follow directions regarding system 
features and use. System customization by different departments for unique 
purposes is possible based on standards and template options 


System functionality and 
usability 


TaskStream is relatively easy to access (logon) and provides a comprehensive 
view of system options on the entry screen. Instructors and students can easily 
determine and select user profiles and types, as well as system security features. 
Both instructors and students can easily create, organize, and upload assignments 
and work online, as well as provide corresponding instructions and assignment 
details. 



Open Source Portfolio Initiative (OSPI) 



The OSPI follows a three-step process (enter, share, view) for portfolio development. This simple sequence 
makes the system useable, but many will find themselves constrained by inflexible organizational capabilities. 



Criteria 


Results 


Planning and goal setting 


Not available 


Creativity ability 


There is little flexibility permitted for individuals. The default version 1.5 only 
permits individuals to show or not show a specific element. There is one template 
available. 


Facilitated communications 
between instructor and 
student 


Commenting features are enabled by default, offering opportunities for students 
and peers to comment on work. The notification system doesn’t work - at least 
not on the Macintosh using Mozilla or IE. 


Collaboration tools 


The only collaboration tools available are the commenting tools. 


Support for reflective 
processes 


Support for reflection is adequate. With customization by the institution, models 
of reflection can probably be provided and edited by individuals. 


Connection and linking 


Connection and linking capabilities are “buggy.” The material manager permits 
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capabilities 


users to upload and organize a variety of file types and URLs. These can then be 
attached to various portfolio elements. However, viewing these artifacts in 
preview mode generates error messages in both IE and Mozilla on a Macintosh. 


Organizational flexibility 


The default installation of the OSPI offers no flexibility for individual users. 
Because the system is Open Source, institutions with sufficient technical expertise 
on staff can substantially customize the system, but these customizations are 
unlikely to result in a highly customizable system for the end user. 


Display flexibility and 
transportability 


The system purports to offer the capability of downloading user portfolios. The 
first author couldn’t make this work on her Macintosh. IE wouldn’t allow me to 
download. Mozilla allowed the download, but then the resulting zip file would not 
extract. 


Data and information 


Not available. 


Start-up costs and 
maintenance 


Start-up and maintenance costs will vary widely depending on an institution’s 
implementation of the portfolio initiative and the in-house expertise for 
maintenance and configuration. Specific hardware requirements will need to be 
satisfied. 


System functionality and 
usability 


Adding an attachment in the personal information section caused the Apache 
server to generate an exception report in both Mozilla and IE. 
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Chalk and Wire 



One of the e-Portfolio system’s evaluated based upon the above criteria was Chalk and Wire 
( www.avenet.neth Although Chalk and Wire has a relatively user friendly e-Portfolio system, some aspects of the 
system were not effective. 



Criteria 


Results 


Planning and goal setting 


Due to the structure of the Chalk and Wire system, students’ portfolios could be 
easily evaluated based upon the desired standards. 


Creativity ability 


Chalk and Wire did not provide very much flexibility for student creativity. 
Students can select templates based on a few standard system templates or upload 
their own. Students can select photographs and illustrations from a few system 
standards or upload their own. Students are unable to move text boxes or display 
assignments or work products except in the format prescribed by the system. 


Facilitated communications 
between instructor and 
student 


Chalk and Wire does offer an internal messaging system for direct communication 
between the instructor and students. 


Collaboration tools 


Chalk and Wire does allow students to share their e-Portfolios with other students. 
Based upon the level of access provided, students could collaborate to create a 
portfolio assignment for a given class. 


Support for reflective 
processes 


Chalk and Wire provides a text box for student reflection. Students can upload 
additional documents to support their reflective comments. 


Connection and linking 
capabilities 


Although Chalk and Wire offers opportunities for providing links to other online 
work, as well as uploading work, limited interaction can be created between and 
among work products. 


Organizational flexibility 


Chalk and Wire offeis limited flexibility for reframing portfolios for a variety of 
applications (e.g., school, job search, etc.). 


Display flexibility and 
transportability 


Student portfolios are created and maintained on the Chalk and Wire system. 
Although the uploads and links could be transported to other e-Portfolio systems, 
students would basically be re-creating their portfolios on the new system. 


Data and information 


Chalk and Wire produces a variety of formatted, printable reports. 


Start-up costs and 
maintenance 


Chalk and Wire limits students’ ability to associate, set-up, and edit course 
profiles. The help system is useful, however, not as robust as many users may 
need. The directions regarding system features and use are clear, however, not as 
comprehensive as may be desired. System customization by different departments 
for unique purposes appears limited. 


System functionality and 
usability 


Although it is relatively easy to logon to Chalk and Wire, logging off and 
determining which system’s options are available for use are not readily apparent. 
Selecting user profiles and types, and determining system security features, also 
are not easily identified or applied. However, students can easily create, organize, 
and upload work online, as well as submit assignments online. Also, instructors 
can easily create assignments online, providing corresponding instructions and 
assignment details. 



ePortfolio Resources 

This final section of the paper contains web addresses and brief descriptions of some of the best Internet 
resources on the subject of electronic portfolios. Links to publications and organizations are included. 

ElectronicPortfolios.org at http://www.electronicportfolios.org/- Compiled by Dr. Helen Barrett, an expert 
on electronic portfolios, this web site offers links to presentations, papers, and how-to guides to develop electronic 
portfolios using generic tools. Of particular interest is the link to the Apple Learning Interchange exhibit at 
http ://ali . apple .com/ ali sites/ali/exhibits/ 1000156/ . 

National Learning Infrastructure Initiative at 

http://www.educause.edU/nlii/kevthemes/eportfolios.asp#resources identifies electronic portfolios as a key theme. 
Their web page links to a variety of references and resources on the topic, including conference presentations. 

ePortConsortium.org at http://www.eportconsortium.org/ - A white paper published in November, 2003, by 
this consortium of institutions is downloadable from their site. Additionally, the References page of their web site 
links to a variety of eportfolio projects, software, papers, and activities. 
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Electronic Portfolio Action Committee (EPAC) Virtual Community of Practice (VCoP) at 
https://worktools.si.umich.edu/workspaces/dcamrid/002.nsf - The EPAC VCoP offers a forum for scholars, thinkers, 
and innovators to discuss electronic portfolios in higher education. The American Association of Higher Education 
(AAHE) sponsors this VCoP. Organizational membership is required. 

Also from the AAHE, the Portfolio Clearinghouse at http://www.aahe.org/teaching/portfolio db.htm 
provides a searchable database of electronic portfolio projects from a global audience. 

Sponsored by the Carnegie Foundation, the Knowledge Media Laboratory at 
http://kml2.carnegiefoundation.org/html/gallery.php offers 25 examples of teaching portfolios created by K- 12 
teachers and those in higher education. Examples come from a wide variety of disciplines. 

Summary 

Eportfolio implementers face a number of important decisions. The system selected impacts both the 
pedagogical and technological environments at the institution. Carney’s (2002) dilemmas illustrate the difficulty of 
the decision-making process for instructional leaders. Shulman ( 1998) also warns about five dangers of using 
portfolios: they become mere exhibitions; they are difficult to complete; the consequences of trivialization; the 
possible perversion of the process; and the possibility of misrepresentation. Instructional leaders must guard against 
these dangers and continue to advocate for the pedagogical goals of electronic portfolios or risk being overtaken by 
high-stakes assessment and accreditation goals of administrators, accreditation organizations, and evaluators. 
Furthermore, the trend to aggregate data carries ethical considerations and offers potential for abuse that must be 
weighed carefully. 

Technology leaders have an equally imposing set of challenges to face. The lack of defined standards in the 
industry pose special challenges for ensuring scalability and integration with legacy systems. The Open Source 
movement offers a viable, if non-traditional, option for institutions with substantial in-house expertise. 

A number of customized systems are available, at a price, to facilitate a non-technical user in creating and 
assessing an electronic portfolio. The electronic portfolio industry is still in a nascent stage, so one can expect a 
number of significant changes to both commercial and non-commercial offerings. It is essential curricular, 
pedagogical, and administrative leaders meet with software developers to educate them about the how eportfolio 
systems should be developed. The OSPI and EPAC communities encourage scholars to become involved and guide 
their development efforts. Curriculum and pedagogy experts must involve themselves in this discussion to influence 
eportfolio development efforts. 
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Abstract 

How did we build a site that has grown to more than 6,000 Web pages and 50 million hits per year - with a 
few part-time people, with no budget for the first five years, and a minimal budget the next five? We will describe 
an efficient and effective design process fundamental to our strategy that is essentially a method of disciplined 
inquiry. Our own XML content management system perpetuates our Website into the future as Web browsers, 
HTML, CSS, and editors continue to evolve. 

Current School of Education Site at Indiana University 

In May, 1994, the first Website for the School of Education at Indiana University Bloomington was 
launched. The home page had two links - one to our Instructional Systems Technology Department, and the other 
to the IUB home page. 

Now our site ( http://education.indiana.edu ) consists of more than 6,000 Web pages, and we have received 
more than 120 million hits in the last two and a half years. Our Website is highly ranked in Google searches. For 
example, searching for the terms, ‘school of education’, in Google has consistently ranked our home page in the top 
five out of about 12 million pages that contain the words ‘school’ and/or ‘education’ - i.e., 
http://www. google. com/search?hl=en&ie=UTF-8&oe=UTF-8&q=school+of+education . 

How did we make this kind of progress in the past decade with a few part-time people, with no budget for 
the first five years, and a minimal budget the next five? We will describe an efficient and effective design process 
fundamental to our strategy that is essentially a method of disciplined inquiry. Our own XML content management 
system perpetuates our Website into the future as browsers, HTML, CSS, and editors continue to evolve. 

Practical Web Design: An Inquiry -Based Process 

Boling and Frick (1999-2004) have developed and refined and effective and efficient process for Web 
development ( http://www.indiana.edu/~pedagogv/preview ). The process itself is inquiry -based. 

Analyze Your Needs and Those of the Users 

We typically conduct a needs assessment by using multiple sources of information. We often interview key 
stakeholders in the School of Education, which include current students, faculty, staff, and administrators. We also 
interview members of our target audiences, or gain information about their needs through our “gatekeepers”. The 
primary target audiences we try to serve on our Website are prospective students (both undergraduate and graduate), 
current students, faculty, staff and associate instructors, and our alumni. Our secondary target audience is K-12 
teachers. Results from these interviews help us to form goals for our Website. We also interview “gatekeepers” in 
the School - the people who often interact with prospective and current students, their parents, and other members 
of the public. We ask the gatekeepers to list the most frequently asked questions they have received over the pas t 
year, who asks the questions, when, how (phone, e-mail, walk-in), and what the answers are. Examples of 
frequently asked questions: “When can I take the Praxis I exam?” (from students who want to get admitted into our 
teacher education program); “What programs do you have?” (from prospective students); “Can you graduate in 
four years?” (from parents of undergraduate students); “What are the prospects of getting a job when finished with a 
degree?” (from students and parents). Finally, we look at Web statistics to see which of our existing Web pages are 
receiving a lot of traffic. For example, we know that the most frequently accessed Web page link on our home page 
is to “Academic Programs and Departments” and that our search engine is used next most frequently. We also get a 
very large number of hits on our K-12 resources pages, such as lesson plans for teachers, classroom management 
styles, for teens only, resources for teaching reading and social studies. 

Next we sort through the data we have gathered. Often we put items on index cards. Then we do a card 
sort, usually by inviting some people we grab from the hallways (i.e., students or staff who are interested and have 
some free time) to rapidly group cards together with common themes. We label the piles of cards and put rubber 
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bands around them, and in turn start grouping the piles, until we get the number of piles of piles down to 10 or less. 
This is effectively creating the information architecture for the Website, or for a subsite within the overall site, such 
as an academic program area. 

Who is the “we” here? Sometimes it is just the Web Director (first author) and graduate assistants. Many 
times in the past it has been student design teams we have formed in appropriate courses in Instructional Systems 
Technology (1ST). This is a good learning experience for our 1ST graduate students, and it is very inexpensive - 
usually just the cost of supplies and materials, such as index cards and computer printing. 

Paper Prototyping and Usability Testing 

Next the design team creates a set of paper pages, called a rapid paper prototype, containing a sample of the 
content and structure we are proposing for the site. The content structure is based on the information architecture 
we derived from the needs assessment. The paper prototype is typically put into a 3-ring notebook. We write 
numbers next to the hyperlinks (underlined text), and then we have tabbed pages with numbers on them, so that we 
can simulate Web browsing. We then conduct usability tests of the paper prototype by selecting members of the 
target audience. Usually we need to only select 4-6 members of each appropriate group. We then observe these 
people who try to use the paper prototype to answer frequently asked questions that we identified in the needs 
assessment. We ask them to think aloud, and we record the paths they take and whether they find the information or 
not (or where they would look for it if it were in the prototype). We occasionally alter the prototype - sometimes on 
the spot - and continue to test it until we have identified major problems with the design of the information 
architecture. If the problems are severe, we attempt to redesign the paper prototype and conduct another round of 
usability tests. Otherwise, we fix the problems and incorporate the design solutions in our computer prototype, 
which is the next phase. 

Computer Prototyping and Usability Testing 

We do rapid computer prototyping next so that we can conduct further usability tests. Now we need to 
attend to some of the Web elements such as how users will navigate the site, banner graphics (or approximations), 
page layouts, occasional images, etc. At this point, we are not trying to make the design completely finished but to 
get enough of it working on the Web so that we can try it with users. In the past five years, we have made this 
process fairly easy by creating new design templates and by using our EdWeb tools (described below) to build or re- 
build a Website with the page content stored in XML files. We then publish those pages on a temporary computer 
account that we use for testing. The temporary site has no public hyperlinks to it, so the rest of the world does not 
know it exists (nor can it be indexed by search engines, since nothing on the Web links to it). The rapid computer 
prototype often has links in it that go to other existing Websites at our university or within the School of Education. 

We then select a new group of 4-6 users who are representative of each relevant target audience, and we 
conduct usability tests in a similar fashion as described above for paper prototype testing. Users are asked to think 
aloud, and we record their paths for browsing (and also how they use the search engine). We use these data to 
identify further problems with the design, including navigation issues. If the problems are still severe or numerous, 
we will make changes and conduct a new round of usability tests with the modified computer prototype with 
completely new users. When satisfied that we have fixed the big problems with the design - based on our usability 
findings - then we move on to the final production of the site. 

Building the Web Site 

At this point we need to pay attention to numerous details for Web publishing which naive developers think 
of as Web design - i.e., getting final versions of graphics produced so that they look good and load quickly, creating 
and debugging style sheets (CSS), making sure HTML or XHTML is valid, making each Web page “look good” in 
terms of layout, use of white space, inclusion of graphics, etc. We also need to test to make sure our hyperlinks are 
working correctly. Normally we do this in our temporary Web account, so the public is unaware of the new site. 

As we are building and testing the site, we ask key stakeholders to take a look at it, such as our Dean, 
department chairs, and others in the School of Education. Based on their feedback and comments we tweak the site 
further. 

Finally, when we are ready to go “live”, we copy the XML files to the actual production site. We update 
our production versions of design templates (a combination of PHP and XHTML), and then we use our EdWeb 
Tools, described below, to publish the Website. If it is a completely new Website, then we need to add hyperlinks 
on other existing Web pages on the School’s site and also notify other sites of the new site. If it is a revision, 
usually little external change is required since we make every attempt to keep file names the same so that we do not 
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break external hyperlinks to the site or parts of it. We also ask our university Webmaster to direct the university 
search engine to index the new site (or re -index it). 

Then we announce the new Website in ways that are appropriate to the occasion. If the site is brand new, it 
also needs to be registered in places such as Yahoo and Google. 

The “we” here is the Web Director and assistants (authors). 

Maintaining the Site 

Every Website is like a new child. Someone needs to watch over it and help it grow and change. This is 
usually where we bring in the content management person in the School of Education who will be responsible for 
this. Basically, we train this person (often a staff member or graduate assistant in department or office) to use our 
EdWeb Tools, so that she or he can update the content as it changes in her or his area of responsibility. Our 
philosophy is to have people closest to the content do the management of it. They know when something changes 
and the Website needs to be fixed to reflect those changes. These people do not need to have special Web design 
skills. Usually someone who is experienced with e-mail and word processing can learn to use our EdWeb Tools and 
a WYSIWYG editor such as FrontPage or Dreamweaver. They focus on the content, since we have provided them 
with a working information architecture derived from our needs assessment and a design and navigation system that 
has been evaluated with members of the target audience through usability testing. 

EdWeb Tools for Content Management 

The EdWeb Page Maker tools were written in PHP (acronym for PHP Hypertext Preprocessor) in 2001. 
PHP is a server-side scripting language that works within HTML documents to enable generating dynamic Web 
pages on demand. Although there are several other options such as Cold Fusion, Java, and Perl, we chose to use 
PHP for two reasons. First, PHP is free. It was open source and we could find examples out there to modify and 
adapt into own programs. Second, PHP is a combination of simplicity and robustness. For example, programmers 
can use a few simple lines in PHP to retrieve or store data in a database. 

In the School of Education, the content of each Web page is stored in XML format on a secure Web server. 
The EdWeb Page Maker tools combine the XML content with the appropriate local design template to publish a 
static HTML Web page for world to see. The School of Education is a complex organization with many program 
areas and offices. Each unit needs to have customized navigation on their Web pages while maintaining a consistent 
design with the overall School of Education Web site. In order to meet such a need, we designed a Web page 
template for each unit with unique navigation sidebar and footer information (See image 3.1 below). Such a locally 
tailored template can provide an effective and efficient way of managing the Web pages within that account. The 
content providers in each unit manage (edit) the main body section of the Web pages. They can create a new Web 
page or modify the content of an existing Web page. Whenever there is a change in the template, the Web Director 
at the School level will make the change and use the EdWeb Page Maker tool to re -publish all the Web pages with 
the new appearance. Once the templates are changed, republishing takes only a few seconds on the live Website 
(analogous to remodeling a jet airplane while in flight). 
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3.1: Three main sections of a Web page 



The EdWeb Page Maker tool is easy to use. Any person who is familiar with word processing software 
such as the Microsoft Word or WordPerfect can use the EdWeb Page Maker tools with minimum training. The 
process of how this tool works is listed below: 

1. Launch Netscape or Internet Explorer. 

2. Click in the location window, and type the URL of the EdWeb Page Maker tool. 

3. The first screen appears as below (See image 3.2): 
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Directions: To edit an existing XML file, click on its name below (second column). To see XML files in a subdirectory, click on the folder name in the 
first column. To upload files, create new XML ffes. new directories, or republish the entire site, see choices in the third column. To find instructions about 
how lo use EdWob Pago Maker Tools for Publishing SoE Wob Pagos. please click on this help link . 



Voritas account = 

Current Unix directory = /ip/iatrfomo/wwws 
Current URL = iKlp: SwS.flpy’&eV- 



List of 
directories: 


List of existing XML 
files: 


Upload a file to your WWW folder (e. 9 .. on Image file. .gif. Jpg. or non-HTML 
document. such as a .pdf. .rtf. or .doc file). 


*.(up») 
gdm n 
admit 
courses 
distance 

& 


admit xml 
avixrr.l 
blife.xml 
calendar, xml 
comself.xml 
courses, xml 
etc. xml 
rWflXSJ.xml 


Create a new XML file at current directory: /jp/Tfltdrjmo/wwws 

Click in the box and type a new filename. 

Examples: Index, history, resources... 


faculty 

forms 

ktA'um 

(stores 

jobs 


finances, xml 
forum.xml 
qoninto xml 
qethere.xmT 
gso.xml 


Creole new !*M_ file 
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private 


fndex.xml 
focoans.xml 
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projects 

soehtml 
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mouein.xml 

nowttua.xml 

newtestxml 


Create a now sub-diroctory under current directory: /rp/futdvmo/Vww»: 
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3.2: The first screen of the EdWeb Page Maker tool 



4. Click on the XML file that you are going to update in the second column. If your XML file is in a 
subdirectory, first click on the subdirectory name in the first column, and then select the XML file. If you 
need to upload a file, make a new XML file, or make a new directory, you can do so in the third column. 

5. If you are updating an existing file, the EdWeb Page Maker tool will open the file and you should see a 
page that looks like this (see image 3.3): 
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EdWebPageMaker Form 

Updating information on istdemo file: gso.xml 

Click m the boxes below to enter or correct any information. 



Author's name |Bude Su 
Document TITLE: (help’) 



J 



About the Graduate Studies Office 



Document MAIN HEADING: (help) 



About the Graduate Studies Office 



d 



Document KEYWORDS: (help) 

admissions, graduate program, instructional systems technology 






3.3: Opens in the editing mode 

(Note: if you are creating a new file, the text input boxes above will be blank.) 
6. Modify fields of the Ed Web Page Maker Web form (see image 3.4). 
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EdWebPageMaker Form 

Updating information on istdento file: gso.xml 

Click in the boxes below to enter or correct any information 
Author's name |8ude Su 



Document TITLE: (help) 



About the Graduate Studies Office 






Document MAIN HEADING: (help) 

About the Graduate Studies Office 



3 

J 



Document KEYWORDS: (help) 



admissions, graduate program, instructional systems technology 






Document CONTENT DESCRIPTION: (hete) 

The Graduate Studies Office handles the admissions process, then - 1 
it comes to the 1ST department. 

d 



Document MAIN BODY' (insert HTML source code that goes between the <BODY>...</Bt 

<h4>Admissions are handled through the Graduate Studies Office. </h4> 

The Graduate Studies Office coordinates the admissions process for all 
graduate programs in the School of Education. You apply through them. 
<p>Your application materials are then sent to the 1ST Department, uhere 
they 

are reviewed by a faculty committee. 

<p>See the <a href*”http://ww». Indiana. edu/~educate/">Graduate Studies 
Office home page</a>. 



Write XML file and preview Web page 



3.4: Editing a Web page 

7. Click the "Write XML file and preview" button at the bottom of the form. You will then see how your 
Web page will look when combined with the template for your department or office (see image 3.5). 
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1ST Demo 

About the Graduate Studies Office 

Admissions arc handled through the Graduate Studies Office. 

The Graduate Studies Office coordinates the admissions process for all graduate programs 
in the School of Education. You apply throuah them. 

Your application materials are then sent to the 1ST Department, where they are reviewed 
by a faculty committee. 

See the Graduate Studies Office home page. 

Return to top 



YOU ARE HERE: tU > Bloomington * School of Education > 1ST Demo 



Indiana University 

School of Education 
201 Worth Rose Ave. 
Bloomington, IN 47405-1006 
25 (812) 856-MSO 



-a 



3.5: Previewing the Web page 



<0 Internet 



8. If the "preview" page looks OK, then click the button at the bottom of that page, and you will publish 
your Web page for the world to see! (see image 3.6) 
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3.6: Publishing the Web page 



40 Interne* 



We have similar tools in the School of Education for updating faculty and staff profiles, for maintaining 
course catalog and syllabi, and for hiding online e-mail addresses of individuals to reduce potential harvesting and 
resultant spam messages. 

In conclusion, the main purpose of using these EdWeb tools is two-fold. First is to centrally control the 
overall appearance of the Web pages so that all the Web pages within the School of Education can have a fairly 
consistent look. Second is to decentralize the content management to the local unit and even to individuals due to 
the fact that they are the people who know what information their Web pages should contain. Of course, such a 
decentralized approach requires certain amount of training for the content manager in each unit. It is usually a one- 
time training session of 1-2 hours in the beginning. Typically each unit has one or two persons to take care of their 
Website (which we consider to be a subsite of the overall School of Education Website). Since the Website 
management in the local unit level does not require a lot of time, the local Web content manager often has other 
duties, e.g., as a secretary or graduate assistant. More information on the EdWeb tools is available at: 
http://education.indiana.edu/guide/guide.html . 

General Strategy for School of Education Web Development 

In summary, the general strategies of maintaining over 6,000 Web pages with three part-time Web 
managers are to: 

• Use an inquiry -based approach to design (user needs assessment, rapid prototyping, and usability 
testing during design and development - which are often graduate student projects in 1ST courses). 

• Keep content in XML format, separate from its appearance on the Web. 

• Have Web managers at the school level design HTML templates and EdWeb Tools in PHP, so that 
content providers focus on content. 

• Let those closest to the content maintain and update it. 
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Abstract 

Interactive activities offer innumerable possibilities within eLearning environments. As such , one 
possibility within the interactive activity realm is netconferencing, so as to support the learning objectives. 
Enhancements within the eLearning environment support the learner’s focus upon knowledge and conceptual 
framework of understanding. Netconferencing supports numerous components that are difficult to create within an 
eLearning environment. As such, netconferencing is an appropriate venue to focus attention towards meeting 
learning objectives. A linkage between aspects will occur. But net conferencing is not a panacea; it is one tool to 
facilitate interactivities within e-Learning. A bridge between what the students already know and need to know still 
needs to be created, and the construction of that bridge may require a number of bricks, concretes, and steel 
scaffolding to make it sound. Inclusion of interactivities that incorporate mental tools such as metaphors, 
inscriptions, narratives, and symbolic representations provide sound groundwork for bridging the gap between 
technology and learning. New technologies, such as net conferencing can then cross the bridge to mediate with 
learners more effectively. 



Introduction 

Interactive activities have the ability to enhance the learning environment, so as to support various learning 
environments that enhance learner comfort levels. Interactive environments, such as netconferencing, have the 
ability to enhance a learner’s conceptual framework of understanding so as to focus a learner’s attention beyond the 
mere technology integrated into the instructional environment, towards building the knowledge and higher order 
thinking skills towards meeting learning objectives. 

To have informed students that understand ideas that are important, 

Useful, beautiful, and powerful. And we also want them to have the 
Appetite and ability to think analytically and critically, to be able to 
Speculate and imagine, to see connections among ideas, and to be 
Able to use what they know to enhance their own lives and to 
Contribute to their culture. (Eisner, 1997, p.349) 

Through the interactive activities made easily available through instructional technology, learners can 
support the creation and enhancement of their conceptual framework, so as to formulate their critical understanding 
and enhancement of the world in which they live, as well as further conceptualize the framework of understanding to 
support the learning objectives. 

How to Bridge the Gap between What Learners Already Know to What They Need to Know 

Representational language and comfort zones Often in the creation of a new medium of knowledge communication, 
such as the Internet, old successful methods of communication are by-passed (Gallini, Seaman, & Terry, 1995; 
Salomon, 1997; Turbayne, 1962; McLuhan, 1968, 1976). The history of the human race is based not only on the 
achievements of individual minds, but on the recognized forms of representation available to the ancestors that 
enabled them to make their ideas and feelings public through cultural representations. Literacy includes the forms of 
representation that convey anthropological, historical, artistic, and inherently recognizable meaning on multiple 
levels of cognition (McLuhan, 1968, 1976; Salomon, 1997; Vygotsky, 1935; Wertsch, 1985). Current iconic 
representations and metaphors used in technology are rapidly taking on grass-roots recognition among peoples of 
many cultures and languages, such as the icons of paintbrushes, houses, worlds, and file folders. So, to assure this 
evolving integration with existing cultures, the “old” knowledge must converge with the new (McLuhan, 1968, 

1976) and the new representations and language will necessarily contain the “old.” Emmanuel Kant suggested a 



192 




close connection between a human’s experience of the world and the inner structure of the mind. (Rothstein, 1996). 
Wolfgang Pauli proposed that 

The process of understanding in nature, together with the joy that man 
feels in understanding, i.e., in becoming acquainted with new knowledge, 
seems to rest upon a correspondence, a coming into congruence of 
preexistent internal images of the human psyche with external objects 
and their behavior.” (Rothstein, 1996, p.203). 



Mediational Tools and Proxemics The traditional dictionary meaning assigned to the word “tool” is “anything, 
which, held in the hand, assists a person to do manual or nonmanual work (Houghton Mifflin Dictionary, 

1974, p.228). Vygotsky used the word “tool” in a similar context, assigning it meaning in relation to work, and 
including physical tools that would include any type of proxemic devices. But his definition would have also 
included mental as well as physical tools. Metaphors, symbols, and semiotic representations of communication are a 
part of the "mental” toolkit, and his definitions would likely, under that definition, also have included technological 
instruments, computer hand devices, robotics and other electronic tools contained in computer devices or electronic 
equipment. Vygotsky offered an alternative, innovative, explanation for mental functions. He suggested that the 
primary tools of activities, represented in signs and symbols, acted as agents for culture, and served as intervening 
links to consciousness (Wertsch, 1985). Mediation of these tools was the structural and genetic central feature of 
mental functioning which became a necessary liaison to consciousness. His seminal research transformed existing 
methodologies with his introduction of this intermediate link of action/object (of study). The body of research 
initiated by Vygotsky offered the basis for a culturally grounded theory of cognition, with the concept of “mediated 
tools” linking culture to the functions of consciousness. 

Tools to Facilitate E-Learning Electronic learning (eLearning) environments transpose space and time. Globally 
people are exposed to the Internet and cellphones with built-in cameras. Modernity (McLuhan, 1976) is 
repositioning time from the linear, past, present and future, and influencing how we interact with others and within 
the construct of our daily lives. Time as a linear descriptor of events is being replaced by a “home page” of 
simultaneously presented information which can be from the present, the past, or predictive of the future. Electronic 
interactivities that can help focus the learner’s understanding of both content and virtual “space” (context), and can 
facilitate learner’s mediation with the knowledge materials presented to the learner. The intention is to help the 
learners move from isolation toward collaborative and community-oriented goals and to do so using as many tools 
that mediate with the learner as necessary. 

E-Learning environments are enhanced through interactive activities that support the learner’s levels of 
motivation within eLearning communities. Vygotsky emphasized the importance of social interactions, such as 
interactive activities within eLearning environments, as imperative aspects related to the progression of the learner’s 
understanding of the subject matter (Vygotsky, 1935; Vygotsky, 1962; Vygotsky, 1978; Vygotsky, 1981; Wells, 
1996; Wells & Chang-Wells, 1992; Wertsch, 1985). 

But Whatever Happened to Crayons? Tactile, recognizable tools, like the tools of play, such as crayons, can still 
conjure up all kinds of memories for almost anyone over 15 or 16 years old, despite all of the movements towards 
the use of technology over those same last 15 years. Using tools that are recognizable, especially those that might 
evoke happy memories or cultural associations, can expand the learner’s sphere of understanding. Vygotsky (1935, 
1962, 1978, 1981) purported that the spontaneous activity of “play,” mediated with the learner and the mental 
processing of information, can produce evolutionary learning from those familiar and spontaneous interactive 
processes. Often in instructional design, however, it can be easy to overlook these same activities and evoked 
cultural memories in favor of electronic tools and “du jour” learning approaches. Interactive processes within e- 
Learning environments often utilize and expand on the electronic offerings available to facilitate the desired learning 
objectives, and these certainly can accomplish the intended results too. Therefore, a look at eLearning activities can 
provide further insight into opportunities to assist student learning. 

Interactive Activities Interactivity within eLearning environments has been defined as “reciprocal events that 
require at least two objects and two actions. Interactions occur when these objects and events mutually influence one 
another” (Wagner, 1994, p.8). Interactivity is a complex structural and conceptual event within eLearning 
environments. Focus upon interactive activities by numerous researchers (Moore, 1989; Hillman, Willis & 
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Gunawardena, 1994; Burnham & Walden, 1997; Crawford, 2000; Crawford, 2003) offered an integrated level 
towards interactive activities when focused upon interactive activities within eLearning environments: learner- 
content (Moore, 1989); learner-interface (Hillman, Willis & Gunawardena, 1994); learner-instructor (Moore, 1989); 
learner-learner (Moore, 1989); learner-self; learner-community (Burnham & Walden, 1997); instructor-community; 
instructor-content (Crawford, 2000); instructor-interface (Crawford, 2000); and, instructor-self (Crawford, 2000). 
Each aspect of interactive activities focus upon enhancing the learning environment, so as to enhance the learning 
objectives. 

Some everyday electronic interactivities that can encourage student mediation include 

• Surfing the Internet 

• Webboard chatrooms 

• List-servs and weblogs (BLOGs) 

• Instant messaging 

• See You, See Me (CUSeeMe) technologies 

• Streamed videos 

• Web phone conferences 

• Net meetings 

• Emerging technologies 

The first two interactivities are already frequently used in e-course websites. List-servs and weblogs are 
also frequently included in e-courses. On the other hand, streamed video is live or downloadable taped video, but it 
still has significant bandwidth requirements that may prove too large for most university servers. If the streaming is 
available or posted on an off-site Webserver (and there are a number of companies that provide this service), 
streaming could prove more viable. The next two interactivities are relatively new and may or may not be available 
through the universities offering the webcourses, but are available in fairly inexpensive versions. (More will be 
discussed about net conferencing in the next section of this paper). 

There are even shareware versions of net meeting and webphone, but there may be online time charges by 
the independent service providers (ISPs) for some of these-type of services. The CUSeeMe technologies require the 
software installed to run the technology on the computer, a small “eye” camera is set up on the computer and sends 
out the live transmission and one on every computer receiving the transmission. 

One specific environment through which to emphasize instructional technologies and interactive activities 
within the learning environment is through netconferencing. Net conferencing offers the ability to enhance the 
eLearning environment through the appropriate and successful instructional design of the learning environment, 
through the possible integration of net conferencing components. Such net conferencing components include other 
electronic interactivities, those of instant messaging, group conference call listings, web phones, whiteboards, access 
to relational databases, and similar environments through which to enhance the learning objectives. 

With the Internet, and ever newer technology, “the unceasing relocation of information in time and space 
(have changed). ..the co-ordinates of time and space have vanished” (Stevenson, 1995, p.106). This “simultaneous” 
relocation of information is generating a more internationally based public sphere that exchanges information across 
the boundaries of nations, hierarchies, and will, ultimately create a new culture of communication and interactivity. 

Leap Frog from Face-to-Face to eLearning 

The leap between face-to-face learning environments and eLearning interactive activities are simplified 
through the integration of net conferencing components, but there are still other eLearning components that work 
well too, but perhaps work better in combination with several technologies. Each aspect integrated into the 
netconferencing tools groups offers the learner, through the expertise of the instructional designer and instructor, a 
bridge through which to build an understanding of the learning that is to occur. The enhancement of the eLearning 
environment, with the focus upon learning objectives, is merely bridged so as to support the learner’s conceptual 
framework of understanding. Other technologies, such as electronic chats, white boards, list-servs and e-mails can 
be effective to a certain degree, depending on the technological and psychological sophistication level of the 
learners. 

Research up to the present indicates there is no significant difference in the learning of students who use e- 
Learning from those in brick and mortar traditional learning environments (Blackwood & Trent, 1968; Bonk, 

Kirkley, Hara, Dennen, 2001; Davis, 2001; Dillon & Gabbard, 1998; Institute for Higher Education Policy, 2000; 
Moore & Kearsley, 1995; Phipps, R., & Merisotis, J., 1999; Russell, 1999). The reality, however, is that e-Learning 
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can be much more challenging initially without some kind of a “bridge” that links what students already know with 
what they need to know (Crawford, 2001;Gannon-Cook, 1998; Vygotsky, 1935, 1962, 1978, 1981; Wertsch, 1985). 

In viewing best practices, bridging to link students’ existent knowledge with what they need to know can 
be done using a variety of the tools discussed in this paper, that is, representational, mental tools, and technological 
tools. The former, mental tools, are more inherent, but still need to be thoughtfully planned and interjected not only 
at the beginning of the instruction, but throughout the module, course, or e-book. The latter, technological tools, 
require in-depth planning to be sure the tools fit with the sponsoring institution’s existent technological architecture, 
and to be sure the tools also fit the subject matter, the philosophical values and delivery styles of the department or 
organization offering the instruction. The technological tools also have to consistently reflect the messages the 
subject matter is designed to convey, with the mental tools interspersed throughout the technology, using the 
representational language, metaphors, inscriptions that imbue the course and university’s messages. The 
technological tools also have to be designed to be truly interactive, utilizing a variation of mediation tools, such as 
the interactivities referenced in this paper. 



Conclusion 

The newer technologies have been referenced in this paper that can affect communication and learning in e- 
Learning. One possibility within the interactive activity realm is netmeeting, an electronic tool that could offer 
inexpensive mediation that could be tried and “tooled” without jeopardizing existent eLearning environments. But 
there are exciting possibilities within the eLearning environments with combinations of eLearning tools and the 
incorporation of semiotic, symbolic, tools as well, that may offer even greater potential for learners. It is this aspect 
that prompted Aristotle to say, after praising Euripides 

at the expense of Aeschylus: ‘The greatest thing by far is to be a master 
of metaphor’ . . .The great sort-crossers from Pythagoras through Plato, 

Descartes, and Newton to Einstein have changed our attitudes to the 
facts. How have they done this? ...(through metaphor). They 
accommodate the feature of attitude-shift... an effective metaphor acts 
as a screen through which we look at the world; or it filters the facts, 
suppressing some and emphasizing others. It brings forward aspects 
that might not be seen at all through another medium. 

(Poetics, 1459, cited in Turbayne, 1970, p.21) 

The old-fashioned “crayons” of the mind, such as metaphors, pictures and other symbolic representations, 
can also have a profound affect on learners, if we choose to utilize them in a careful design for e -Learning. 

Enhancements within the eLearning environment support the learner’s focus upon knowledge and 
conceptual frameworks of understanding. A linkage between electronic interactive activities and semiotic mediation 
is the focus of this discussion and one that needs to remain “inside the lines” of instructional design. 
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Numerous models of the instructional design process have been created as a guide for those who lack 
expertise in the process. Instructional design (ID) models, like all models, are simplified representations of reality, 
often created as a means of representing the schemata that experts use to solve problems in a way that is usable and 
understandable to a novice. 

Almost all classic instructional design (ID) models are a variation of the “ADDIE” (Analysis, Design, 
Develop, Implement, and Evaluate) model (Smith, & Ragan, 1999). Typically, the Analysis phase includes needs 
assessment, goal and subordinate skills analysis, learner, context, and task analysis, and goal identification; the 
Design phase includes identification of performance objectives, creation of test items, and development of 
instructional strategies; the Develop phase includes the development of the instructional materials and management 
strategies; the Implementation phases includes planning and managing the implementation of the instruction; and the 
Evaluation phase includes formative and summative evaluations (Lohr, 2003). 

Although it is generally accepted that instructional design involves an iterative cycle where “inevitably, 
working on one design activity leads to implications or solutions for other design activities” (Smith & Ragan, 1999, 
p. 8), Branch (1997) found that novice designers perceive instructional design as an inflexible and lock-step process, 
due in part to the linear manner in which most ID models are represented visually. The linear presentation of classic 
models such as ADDIE “emphasize[s] closure of each component in the process to serve as input to the next 
component.” (Tessmer & Wedman, 1990, p. 80). 

Problem-solving behaviors are influenced by the mental models that designers hold (Gagne & Glasser, 
1987). According to Rowland (1992), when presented with an instructional design problem, expert designers recall 
a template or mental model for the type of information needed to solve the design problem (Rowland, 1992). 

Students who are taught to design instruction using a linear model may find it difficult to adapt to the realities of 
instructional design practice where “inevitably, working on one design activity leads to implications or solutions for 
other design activities” (Smith & Ragan, 1999, p. 8). In the best of circumstances, novices often experience 
cognitive dissonance as they try to reconcile the way they were taught with the recursive nature of instructional 
design practice. 

This presentation will discuss lessons learned when an instructional designer shifted from the traditional 
ADDIE model of instructional design to the more flexible “Practitioner’s model” (Kalk & Cennamo, 2003), as they 
were applied to the Learning Technologies and Health Care (LTH) initiative, jointly sponsored by Harvard Medical 
School, the Technology in Education Program at the Graduate School of Education at Harvard University, and The 
Multidisciplinary Surgical Intensive Care Service (ICU) at Brigham and Women’s Hospital. 



Application of the ADDIE model 

The LTH initiative began in luly of 2002 with a project manager, funded by a small grant, and a volunteer 
team of physicians, nurses, fellows, and faculty. The overall goal of the LTH initiative is the development, 
implementation, and evaluation of educational tools, such as interactive multimedia learning modules, for medical 
education in an intensive care setting. 

The project manager implemented the ADDIE model to organize the approach to the broad task of 
improving critical care education. As a result, the design steps of the ADDIE model were implemented over a one- 
year period (see Figure 1), beginning with a thorough analysis of the current education offerings in the ICU 
environment, as well as extensive research, and evaluation of all medical education tools. Overall, the participants 
in the LTH initiative were extremely enthusiastic about the LTH initiative. They had high expectations of the 
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project manager to deliver functional multimedia tools that could be integrated immediately in their fast-paced and 
high-pressure learning environment. 



1 



ANALYSIS 

? Who is the audience? 

? What do they need to learn? 

? What are the delivery 
options? 

? What constraints exist? 

? What will learners do to 
demonstrate competency? 



July-September 



DEVELOPMENT 




DESIGN 




IMPLEMENTATION 


? Writing performance 
objectives. 




? Creation of instructional 
materials 




? Duplicate and distribute 
materials. 


? Sequencing content and 
data. 

? Selecting instructional 
methods: 

Web based 

CD-ROM 

Both? 




? Present information is many 
different multimedia formats so 
that the learner's preference 
can be met. 

? Determine appropriate 
interactions. 

? Create activities for group work 
to help construct a supportive 
social environment. 




? Install and maintain the 
course. 

? Be prepared for technical 
problems and offer 
alternatives to learners. 


September-October 




November - February 




February-April 
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EVALUATION 

? Conduct formative evaluation 
to improve the course. 

? Conduct summative 
evaluations to judge the 
effect of the course. 

? Plan several points during the 
course when learners can 
provide feedback. 

? Test for criterion-related 
items and completion of 
competency skills. 



May-June 



Figure 1. LTH Design Plan using the ADDIE model 

However, problems emerged in the development phase. The ADDIE model began to break down when the 
team’s expectations did not match the project’s limited budgetary resources and the constraints of the timeline. 
Although the designer emphasized the need for continuous interaction and evaluation through the steps, the clients 
were continually referring to the linear model as a benchmark to the progress of the program. In addition, planned 
sources of funding and technical resources did not become available to LTH initiative at the conclusion of the first 
year. 

While the clients were enthusiastic regarding the program, they quickly became frustrated with the lack of 
tangible technology deliverables. Clients grew impatient with the time required to analyze and design the product 
and were frustrated that, despite all their investment of time, they still didn’t have the usable educational tools they 
needed. Meanwhile, the project manager felt unable to move forward with the current expectations, including the 
deliverables, budget and deadlines, which had been planned according to the ADDIE model. As a result, in the 
second year of the program, the instructional designer re-evaluated the design plan and introduced the iterative 
design model (Kalk & Cennamo, 2003). 



The Practitioners Model 

The “Practitioner’s model” of instructional design was developed through a recursive process of examining 
instructional design literature, analyzing design artifacts collected throughout more than 20 years of combined 
experience in corporate and academic settings, attempting to outline and diagram the decisions reflected in the 
design artifacts, and returning to both the data and the instructional design literature for confirmation and to look for 
contrasting cases. Based in theory and derived from practice, this model is designed to be easy for novices to learn 
and to implement so that they can begin achieving success early in their careers. There are three key components to 
the model, each represented by a graphical device 

The Essential Triangle 

Five essential elements form the building blocks of systematically designed instruction: learner needs and 
characteristics, goals and objectives, instructional activities, assessments, and formative evaluation. These five 
Essential Elements are illustrated as a triangle (see Figure 2). Learners are at the center of the “essential triangle” of 
instructional design because they are the focus of the instruction. Outcomes, assessments, and activities are placed 
in each of the three corners to illustrate that they must be in balance, or alignment, for the instruction to be effective. 
Throughout the design process, formal and informal evaluations should be used to monitor whether the instructional 
activities, outcomes, and assessments are in alignment, and whether the outcomes, assessments, and activities are 
appropriate to the needs and characteristics of the learners; thus, evaluation is wrapped around the others elements in 
the triangle. 
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Figure 2. The Essential Elements of Instructional Design 



The Design Spiral 

The model places an emphasis on instructional design as a knowledge-building cycle that can be thought of 
as a spiral with five distinct phases (see Figure 3). Starting at the center of the spiral with the Define phase, and 
moving out through the Design, Demonstrate, Develop, and Deliver phases, designers continuously question, 
communicate, collaborate, and refine. The designer begins in the center of the spiral, the Define stage, and has very 
limited understanding of the project. As designers cycle through the phases, they continually revisit the five 
essential elements, revising their understanding where necessary, adding details, and building knowledge. 




Define 
Design 

\ \ • Demonstrate] 
V ' Develop 
' Deliver 

Figure 3. The Phases of Instructional Design 



Evaluation 



The ASC Cycle 

The ASC cycle gives form to the iterative process used to revisit each of the elements throughout the 
design process (see Figure 4). Each phase begins with asking for, and assembling, information. Designers engage 
with their clients and team members to get information and begin to make sense of the information they have 
gathered relative to the design process. They synthesize the information and generate tentative solutions, then 
confirm their understanding through presenting materials for review. As designers check their tentative solutions 
with others, they gain additional information that will move the project from one phase in the instructional design 
process to the next. Gradually, the designer builds understanding until he or she has a deep and broad understanding 
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that allows the project to be completed and delivered. 



Check and 
confirm 




and solve 

Figure 4. The collaborative ASC cycle 



Application of the Practitioner’s model 

In the second year of the LTH initiative, the project manager moved away from the seemingly linear 
ADDIE model to Practitioners model. The model emphasizes the collaborative nature of instructional design work, 
which requires designers to be skilled in negotiation, brainstorming, problem-solving, communication, and project 
management. Most instructional designers work as part of a team, which can include Subject Matter Experts 
(SMEs), clients, project managers, graphic artists, media producers, programmers, and quality assurance specialists. 
The model encourages designers to advocate for the instructional design function on their team, educating clients 
and team members about the process and value of instructional design. 

Additionally, and critical in the context for this paper, the model stresses the need to make pragmatic 
decisions in order to ensure success within the constraints of real budgets, deadlines, and other practical pressures. 
This can mean moving ahead in the design process without complete information, such as starting work on 
assessment before finalizing the objectives, knowing that the spiral nature of the work will allow for opportunities to 
revisit the objectives, and conplete them, in subsequent phases. 



The ASC cycle in action 

The ASC Collaborative Cycle was applied as a starting point to shift the clients to the new instructional 
design model, and to engage the clients in the design process. Using the ASC cycle, the designer collaborates with 
the client, SMEs, and learners to create progressively more complete versions of the product. At each phase, 
designers assemble information, synthesize tentative solutions, and confirm or modify those design decisions 
through presenting tentative ideas and products to the client, SME, and learners in order to further refine the 
designer’s thinking about the learner’s needs and characteristics, the desired learning outcomes, potential 
assessments, and the instructional product. 

In this project, the designer initially confirmed the team’s understanding of the design process and the 
switch from the ADDIE model timelines to the practitioner spiral model. She became an advocate for the learner by 
both educating and engaging the client in the design process. Throughout, she worked to set attainable short-term 
goals with the team and managed their expectations of the tangible deliverables of each phase. Figure 5 shows how 
the designer implemented these strategies using the ASC cycle. 
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The Design Spiral phases in action 

The project manager then introduced the Design Spiral (Kalk & Cennamo, 2003) model. Using this model, 
instruction is designed in a “layered” fashion, with the designer spiraling through the essential questions of design, 
considering the elements of learners, outcomes, assessments, activities, and evaluation at increasing levels of 
specificity as they move through the phases of Define, Design, Demonstrate, Develop, and Deliver. Although the 
output of one phase in the model serves as the input for the next phase in the model, this system differs from 
traditional instructional design systems. In traditional instructional design systems, an analysis of the learner’s 
needs leads to the identification of goals and objectives; the identification of goals and objectives lead to the 
development of assessment items and instructional strategies, instructional strategies are determined before media is 
selected, and the product is developed before it is evaluated (see, for example, Dick, Carey & Carey, 2001). In this 
model, each of these “essential elements” is considered in each phase of the instructional design process. The 
phases are distinguished not by the element addressed in each phase, but instead, by the depth of information 
gathered and synthesized for each element. 

Beginning with phase one or the Define phase, the project manager redefined the goals of the project. The 
initial emphasis of the LTH initiative was to create high end, and sophisticated, multi- media learning tools. Since 
the group’s resources and funding were scarce to effectively achieve this goal, the focus shifted to the more 
attainable goal of developing and testing a new learning model that could then be used as a framework for various 
technological learning tools. As a result the broader “Medical Performance Support System” or MPSS was 
conceived and served as the focus of the program. In addition, the project manager went back to the research on 
Electronic Performance Support Systems as a guide to develop new the MPSS infrastructure. The overall research 
question of the program was also redefined at this phase to state, “ How can a premier acute care hospital address 
the goals of high quality care and patient safety in a rapidly evolving healthcare environment?” Finally, the group 
developed a comprehensive evaluation plan to address this research question. Specifically, evaluation tools to 
measure learner’s proficiency and the variability of ICU procedures were developed. As a result, online surveys, 
focus groups, interviews and observations were developed. Each of these evaluation tools would be administered 
repeatedly throughout the entire design process. Table 1 illustrates how the learners, outcomes, activities, 
assessment and evaluation were addressed in the Define phase. 

Table 1. The Define Phase 



Learners 


The learners were defined as nurses, residents, fellows and 
physicians in intensive care environments 


Outcomes 


Went back to needs analysis and redefined project goals from 
developing multimedia tools for intensive care environments to 
developing a new educational model and learning strategies for 
intensive care environments: 
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The goal of this project is to develop a medical performance 
support system; namely, a technology infrastructure that will help 
ensure patient safety, facilitate standardized practice and foster 
professional development 

Activities Went back to the research on EPSS systems and more appropriate 

distance learning tools. Focused project to a “medical 

performance support system” than simply an eLearning tool. 

Assessment Developed assessment instruments to determine both learner’s 

proficiency and the variability of ICU procedures. Developed 

surveys and focus groups, observations, and interview protocols. 

Evaluation Redefined research question: 

How can a premier acute care hospital address the goals of high 
quality care and patient safety in a rapidly evolving healthcare 
environment? 

Developed research and evaluation plan. 

Phase two, or the Design phase, was then implemented. Since the main goal of the project had shifted, 
high-level objectives were redefined. In addition, the technical proficiency of the learners was analyzed based on 
preliminary surveys of the target audience of medical students, nurses, fellows and physicians as well as the 
technical infrastructure of the hospital. As a result, the project manager began to test and evaluate different 
technical tools that would match the revised objectives of the MPSS program. Table 2 illustrates the various design 
strategies of the Design phase. 



Table 2. The Design Phase 



Learners 


Technical proficiency of learners was evaluated utilizing an online 
survey. 


Outcomes 


Redefined high level objectives: 




• 


The learner will be able to access all MPSS learning tools 
“on demand’’ and at “bedside”. 




• 


MPSS learning modules will ensure standards of “best 
practices” to improve patient safety. 




• 


MPSS learning modules will be based on instructional 
design process and theories. 




• 


MPSS learning modules will maintain competency of 
high risk, low volume procedures. 




• 


MPSS learning modules will be learner-centered tools 
that are valuable to different levels of experience and 
expertise. 




• 


MPSS will facilitate evidence-based learning. 


Activities 


Testing of new different technical tools based on results of 
evaluation of learner’s technical proficiency and hospital 
infrastructure. 


Assessment 


Repeated assessment measures conducted in the define stage 
throughout such as surveys, focus groups, observations and 
interviews. 


Evaluation 


A research plan was developed at this stage to help identify 
themes that emerge from preliminary surveys. 
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The project manager increased the number of instructional design products in the third phase or the 
Demonstrate phase. Additional storyboards, wire frames, and prototypes using lower level technology enabled the 
group to understand the design process better, and provided meaningful feedback. For example, initial technical 
requirements for the LTH learning modules required higher-level web browsers and shockwave plug-in to play. 
Since the program objectives were redefined, lower level prototypes were developed using Microsoft PowerPoint to 
test the instructional effectiveness of new MPSS learning model. Table 3 outlines the strategies employed in the 
Demonstrate phase in greater detail. 



Table 3. The Demonstrate Phase 



Learners 


Revisited audience characteristics and shifted technical platforms 
for prototype to ensure usability in the hospital environment. 


Outcomes 


Based on both evaluation of data and research the project manager 
shifted from Macromedia Director to Microsoft PowerPoint as the 
technological platform to create multimedia deliverable to ensure 
consistent interface and that learning objectives were met. 


Activities 


Provided instructional design products. 

• Provided more “slices” - using wire frames and storyboards 

• Developed prototype in lower level technology platform to 
simulate interactions and learning goals 

• Developed presentation CDs of to aid in goal of obtaining 
additional resources and increase testing 


Assessment 


Repeated assessment measures such as surveys, focus groups, 
observations and interviews. 


Evaluation 


Continued research and evaluation plan 



During the fourth phase or the Develop phase, the project manager was able to conduct several pilot testing 
opportunities within the hospital setting. In addition, the project manager presented the results of the MPSS 
research efforts at medical conferences and to potential ICU education stakeholders such as hospital 
administration and ICU medical device manufacturers. Utilizing the instructional design projects created in the 
Design Phase, the project manager was able to show the holistic approach to the design process, and how the 
MPSS infrastructure would effectively address ICU education. Because the technical limitations were resolved, 
nurses, residents and physicians were able to interact with the prototypes of learning tools more readily and 
provide immediate feedback on the value of future MPSS tools. As a result of the various pilot testing and 
presentations, additional funding was received, and future plans for commercial MPSS sustainability were 
created. Table 4 below shows the various strategies that were explored in the Develop phase. 

Table 4. The Develop Phase 



Learners 


To obtain additional funding and support, the team focused on 
hospital administration and other stakeholders of ICU education. 


Outcomes 


The presentation of the design process and prototype of 
multimedia deliverables increased funding for further 
development of tools and hospital support of research efforts 


Activities 


Based on the feedback of the multimedia deliverables a plan to 
create additional learning modules was created as well as the 
development of an external website for MPSS 


Assessment 


Repeated assessment measures such as surveys, focus groups, 
observations and interviews. 


Evaluation 


Continued research plan More pilot testing and feedback 



The final phase, or the Deliver phase, focused on the transition from the initial LTH grant funded program 
to the future sustainability of the MPSS organization. The project manager concluded her design efforts by 
compiling all of the design materials and created both a project summary and a summative evaluation of the 
program. In addition, detailed plans for the MPSS organization were developed such as the organization’s mission. 
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services, solutions, budget, evaluation and multimedia development plans were created. Finally, these design and 
research efforts were incorporated in MPSS comprehensive commercial website - http://www.medperformace.com. 
Table 5 shows the various strategies implemented by the project manager in the Deliver phase. 



Table 5. The Develop Phase 



Learners 


The potential learners of the MPSS organization was expanded to 
include the additional ICU stakeholders identified in the Develop 


Outcomes 


The mission of MPSS was formulated: 

To combine best practice guidelines with innovative technology in 
the development of educational solutions designed to support 
quality and safety in the healthcare environment, 




And the MPSS commercial website was launched 
http ://w w w . medperformace .com 


Activities 


The following facets of the MPSS organization were clearly 
defined: 

• Services 

• Detailed list of potential “solutions” 

• Multimedia learning module design plan 

• Sample operations and development budget 




All project materials were compiled. 

A summary of the design process was created. 


Assessment 


Suggested modifications to assessment instruments resulting from 
formative evaluation were discussed for future MPSS clients. 


Evaluation 


A summative evaluation was conducted 



Conclusion 

All in all the iterative design model helped reshape the scope of the LTH initiative and provided the clients 
with more tangible and effective learning tools. As a result the LTff initiative received additional funding and the 
“Medical Performance Support System” organization was formed. As a result of implementing this model, the LTH 
initiative was able to move forward and to acquire additional resources. More importantly the following lessons 
were learned: 

• The design process is an iterative and not a linear process. 

• The needs analysis never ends. 

• Clients need constant education of the instructional design process to stay motivated. 

• Timelines need to be revised immediately if they are not realistic. 

• Providing clients with “slices” or “prototypes” using lower end technology can help illustrate learning 
goals while demonstrating desired technical capabilities of the final products. 

We believe the Practitioner’s model was more effective because it was 
o More holistic 
o Flexible 

o Iterative - engaged the clients more 
o Increased collaboration 

• Moved the project forward and both designer and client are happy 

Recently, Schatz (2003) suggested that instructional designers begin to openly reflect on and talk about their 
design practice in order to learn from our successes and our failures. “It is by looking within the processes that we 
as designers of instruction go through while making crucial design decisions that we may develop the potential for 
expansion and evolution of our field’s practice. If we can investigate, discuss, and reflect upon these design 
decisions and evaluate decisions in light of results, we have a powerful tool for guiding and improving practice.” 
(Schatz, 2003, p. 60). 

How can this model be incorporated in your ID setting? 
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Internet Searching by K-12 Students: A Research-based Process Model 



Kathleen Guinee 

Harvard Graduate School of Education 

Abstract 

Understanding K-12 students’ Web-based research practices and the challenges posed by this task , can 
help educators assist students with learning to manage this complex process. In this presentation, I propose a 
research-based process model of K-12 students ' Internet search practices. The model describes the ways in which 
students search for, evaluate, and use information on the Web and suggests strategies that students can develop to 
improve the effectiveness and quality of their Web-based research. 

For United States students, conducting research using “a variety of technological and information 
resources,” including the World Wide Web has become the norm (The National Council of Teachers of English, 
2000). The 59% of students ages 5-17 who used the Internet in 2001 reported that their number one use of the 
Internet was for schoolwork (National Center for Education Statistics, 2003). This widespread utilization of the 
Web for academic (and vocational) purposes necessitates understanding students’ Web-based search processes in 
order to assist them with learning this complex process. 

During the past decade, several research studies have been conducted to investigate K-12 students’ Internet 
research practices. This paper presents a process model summarizing this research. The model describes the ways 
in which students search for, evaluate, and use information on the Web and suggests strategies that students can 
develop to improve the effectiveness and quality of their Web-based research. 

The Internet as a K-12 Research Tool 

Elementary (Kafai & Bates, 1997), middle (Bilal, 2002; Wallace, Kupperman, Krajcik, & Soloway, 2003), 
and high school (Fidel, et al., 1999) students enjoy searching the Web and using it for school-related research 
projects (Eagleton, Guinee, & Langlais, 2003; Jackson, 1996). As students enter higher grades, their use of the 
Internet for schoolwork increases (Lien, 2000). Within various middle and high school curricula, Internet inquiry 
assignments are being incorporated into language arts (Eagleton, Guinee, & Langlais, 2003; Large, Beheshti, & 
Moukdad, 1999), science (Fidel, et al., 1999; Jacobson & Ignacio, 1997; Gordon, 2000; Wallace, et al., 2003), social 
studies (Jackson, 1996; Lorenzen, 2001), and library/media center (Gibson & Mazur, 2001; Hirsh, 1999) classes. 

When conducting research, today’s students use a combination of electronic and print resources (Fidel, et 
al., 1999; Gibson & Mazur, 2001; Large & Beheshti, 2000; Lorenzen, 2001). Some students strongly prefer using 
the Internet as their primary information source (Gibson & Mazur, 2001; Jackson, 1996; Jacobson & Ignacio, 1997; 
Large & Beheshti, 2000). One reason students give for preferring the Web over traditional print materials is that 
they feel they can locate information faster when using the Internet (Large & Beheshti, 2000; Vansickle, 2002). 
Other students decide whether to use electronic or print materials based upon the type of information they are 
seeking (Large & Beheshti, 2000). For example, students tend to favor the Web for locating up-to-date or relatively 
obscure (e.g., information about the sport of curling) information and they favor print sources for obtaining 
comprehensive, organized information on general topics (Large & Beheshti, 2000). Still other students prefer to 
reference only traditional print materials for their research, despite the increasing prevalence of electronic sources 
(Large & Beheshti, 2000). 



Process Model of K-12 Internet Research 

The following model (see Figure 1) outlines the activities students perform while conducting research using 
the Internet and discusses the ways in which students generally perform these tasks. 

Develop Research Question 

Research questions guide students through the process of conducting research (Bowler, Large, & Rejskind, 
2001), helping them to stay organized and on task (Gibson & Mazur, 2001). Unfortunately, middle school students 
have difficulty developing research questions (Bilal, 2002; Wallace, et al., 2000). The research questions they 
generate range from too broad, general questions to too narrow, specific fact hunts, with only about a third being 
focused, researchable questions (Bilal, 2002; Eagleton & Guinee, 2003; Wallace, et al., 2000). Even high school 
students falter when constructing research questions because many feel that they don’t need to plan ahead while 
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searching online (Fidel, et al., 1999). Needless to say, developing specific, researchable questions is an important 
skill to practice with middle and high school students (Eagleton & Guinee, 2004). 
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Figure 1 . Research-based process model of K-12 students Internet searching practices. 

Visit Known Site 

After identifying a research question, one strategy students use to locate information to answer the question 
is to visit a Web site directly. They can do this in several ways. Sometimes, students plug their broad topic into the 
“dot-com formula,” using www . mytopic . com as their URL (Guinee, Eagleton, & Hall, 2003). Other times, 
students visit sites they have used in the past, even if they pertain to a different topic (Fidel, et al., 1999). Students 
also visit sites they suspect might contain their desired information based on their understanding of the research 
topic (e.g., visiting the Coast Guard site to research boat safety) and prior experiences with the Web (Guinee, 
Eagleton, & Hall, 2003). Or, students might visit sites they feel are highly trustworthy, such as those created by 
universities, the government, or news agencies (Lorenzen, 2001). 

Finally, an effective approach students employ for deciding which site to visit is to follow 
recommendations from others. Educators often provide students with a list of relevant URLs or bookmark 
appropriate sites on their classroom computer(s) for students to visit. It’s important to note that primary grade 
students are better able to use bookmarks supplied by the teacher than to type long, complex URLs (Kafai & Bates, 
1997). In addition to educators supplying potential sites, students often share URLs with one another (Jacobson & 
Ignacio, 1997). 
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Use Search Engine 

Using a search engine is another strategy students apply to locate answers to their research questions on the 
Web (Guinee, Eagleton, & Hall, 2003; Lorenzen, 2001). Starting in approximately 5 th grade, students can 
independently locate information using the Web (Kafai & Bates, 1997; Lien, 2000). However, by using scaffolds, 
younger children are also able to successfully execute searches (Kafai & Bates, 1997; Revelle, et al., 2002). 

Students tend to have strong preferences for one search engine or another (Vansickle, 2002). Most use only 
one search engine in a given search session (Large, Beheshti, & Moukdad, 1999; Schachter, Chung, & Dorr, 1998; 
Vansickle, 2002). Interestingly, older students tend to use more search engines than younger students, perhaps 
because they have more experience using the Web and are therefore more aware of different search engines (Lien, 
2000). Having a preferred search engine is fine, as long as students are constructing appropriate queries for that 
engine and are willing to be flexible, occasionally using a different engine when necessary. 

While using a search engine can be a successful method for finding information on the Web, it can also be 
a source of frustration for students. Many students get frustrated by slow response times of search engines and by 
their own inability to locate their desired information in a “reasonable” timeframe (Fidel, et al, 1999; Wallace, et al., 
2000). Students may be able to avoid at least some of this frustration by learning to construct more effective search 
strings. 

Construct Search String 

A predominate difficulty students experience while performing Web-based research is constructing 
effective search strings (Gibson & Mazur, 2001; Large & Beheshti, 2000; Wallace, et al., 2000). In general, middle 
school students demonstrate unsophisticated skills when constructing search strings, using mainly broad terms and 
phrases (Guinee, Eagleton, & Hall, 2003; Wallace, et al., 2000). Several factors contribute to students’ abilities to 
develop suitable search strings, such as their background knowledge, language skills, and computer experience, as 
well as the nature of the task (i.e., open-ended inquiry vs. fact finding and teacher-assigned vs. student -selected 
topics). 

First, possessing sufficient background knowledge is important for constructing search strings (Eagleton & 
Guinee, 2004; Fidel, et al., 1999). Most obviously, background knowledge helps students identify potential relevant 
search terms. In addition, background knowledge helps students conceptualize the information space for their topic 
(Kafai & Bates, 1997). On their own, middle school students tend not to apply an understanding of the topic space 
to develop a plan for their research (Guinee, 2004). However, with direction, middle school students can focus, 
narrow, and broaden their search (Eagleton, Guinee, & Langlais, 2003; Kafai & Bates, 1997). Even high school 
students struggle with conceptualizing the topic space for their query, sometimes omitting required concepts and 
other times adding unnecessary ones to their search strings (Nahl & Harada, 1996). 

In addition to general background knowledge, proficient language skills are important for successfully 
constructing search strings (Eagleton & Guinee, 2004; Nahl & Harada, 1996). Students must be able to specify 
relevant search terms, synonyms for these terms, and related terms (Nahl & Harada, 1996). Sadly, Nahl & Harada 
found that many of the high school students they surveyed did not have sufficient vocabulary skills for constructing 
successful search strings. Students need strong language skills that can enable them to be flexible when building 
search strings, by adding new terms or substituting different terms to rephrase a query. For example, a student 
searching for the amount of money a sports star “makes in a given year,” needs to identify “salary” as a likely search 
term to answer his or her question (Eagleton & Guinee, 2002). 

Finally, students who possess more computer experience tend to construct and interpret search strings more 
effectively than their peers (Bowler, Large, & Rejskind, 2001; Nahl & Harada, 1996). Vansickle (2002) concludes 
that a student’s general computer skills may be related to his or her access to computers. Besides creating an initial 
discrepancy in students’ search skills, differences in general computer skills may escalate over time, causing 
disadvantaged students to be at an even greater disadvantage. For example, Kafai and Bates (1997) observed that 
during group work, students with prior Internet experience dominated the interactions at the computer. 

On average, middle school students conduct one search every 4 or 5 minutes (Large, Beheshti, & Moukdad, 
1999; Wallace, et al., 2000). They use two primary methods to construct the search strings for these queries: 
discrete terms and natural language (Guinee, Eagleton, & Hall, 2003). During teacher-assigned, open-ended 
research tasks, most students search using single discrete terms that describe a broad concept (Bilal, 2002; Guinee, 
2004). Students generally demonstrate a slightly different tactic when they select their own open-ended research 
topic. In these cases, students tend to search using multiple (usually two) discrete terms (Bilal, 2002; Large, 
Beheshti, & Moukdad, 1999), which can be described as the “topic and focus” method (Guinee, Eagleton, & Hall, 
2003). 
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During fact-finding search tasks, students tend to use natural language queries more often than they do on 
open-ended research tasks (Bilal, 2002; Guinee, 2004). In most search engines, the negative consequence of using 
natural language queries, as opposed to discrete terms, is that the additional words in the query artificially constrain 
the possible results. Nevertheless, natural language queries are popular with students. Approximately two-thirds of 
middle school students and one -third of high school students construct fact-finding search strings by using either 
natural language phrases or full natural language questions (Nahl & Harada, 1996; Schacter, Chung, & Dorr, 1998). 
From this observation, Schacter, Chung, & Dorr conclude that students use natural language queries because they do 
not have an accurate understanding of how the Internet operates or how search engines interpret search strings. 

They feel that to become effective searchers, students should have an accurate mental model of the Internet and how 
it works. As an alternate explanation, Guinee proposes that students are failing to abstract the general topic and 
focus from their specific fact-finding research questions. She suggests that even when searching for facts, students 
need to identify the topic space for their query. 

Analyze Search Results 

A seemingly minor, yet important task during Internet research is assessing the list of search results. 
Generally, middle school students have difficulty differentiating good and bad hits (Large & Beheshti, 2000). One 
reason for this may be that students are reluctant to read the results list thoroughly, predominately using the titles to 
determine quickly whether or not to investigate a hit (Kafai & Bates, 1997). It is interesting to note that students 
visit an average of only one hit per search (Large, Beheshti, & Moukdad, 1999). 

Evaluate Site 

Middle and high school students have difficulty evaluating the quality of the information they find on the 
Web (Gibson & Mazur, 2001; Lorenzen, 2001). While one may argue that this difficulty arises from the adult 
reading level of Web content, readability does not seem to be a stumbling block for older students. Only 38% of 
middle school students report having problems with the vocabulary on the Web (Large & Beheshti, 2000). The real 
issue may be one of trustworthiness. Many middle school students believe that all the information on the Web is 
reliable (Schachter, Chung, & Dorr, 1998). 

Alternately, students may be having problems evaluating sites because they’re not spending enough time 
examining them. While searching on the Internet, students make an average of 1 mouse-click every 1.2 minutes 
(Large, Beheshti, & Moukdad, 1999). Given the amount of time it takes for search results to be compiled and pages 
to load, this leaves little time for reading content. In addition, students often scroll through a site too quickly to 
actually read it and rarely follow links to investigate a site further (Wallace, et ah, 2000). Students tend to simply 
glance at a site, often using the quality of graphics as their discrimination criteria, and then return to the search 
results list (Wallace, et ah, 2000). This tendency is perhaps one of the downsides of Internet research. 
Disappointingly, some researchers have concluded that the interactive nature of the Web impedes students’ mission 
to read and evaluate content while conducting research (Large, Beheshti, & Moukdad, 1999; Schachter, Chung, & 
Dorr, 1998; Wallace, et ah, 2000). 

Identify Information 

As mentioned earlier, students get frustrated when they can’t find appropriate information while searching 
(Fidel, et ah, 1999; Large & Beheshti, 2000). Students have mixed-opinions about how easy or difficult it is to find 
information on the Web (Jackson, 1996). However, some factors seem to facilitate students’ success at finding 
information. For instance, more experienced computer users are more adept than their peers at finding correct 
answers to fact-finding tasks (Guinee, 2004). Also, elementary school students who work collaboratively to search 
the Web have greater success at finding relevant information (Lien, 2000). Finally, the ability to efficiently identify 
relevant information also depends on the task constraints and students' perceptions of the task purpose (i.e., to find 
the correct answer). 

Generally, students are more successful at locating relevant information for open-ended research tasks than 
at finding correct answers to fact-finding tasks (Bilal, 2002; Schachter, Chung, & Dorr, 1998). Schachter, Chung, 
and Dorr concluded that students’ success at open-ended research tasks is a result of the simple fact that more 
potential answers exist for open-ended questions. Students recognize that a requirement of successfully completing 
a research task is to locate information on their topic. As a result, students can be so intent on finding information 
for their assignment that they will change their research questions to match information they have already located 
(Fidel, et al„ 1999). 

Take Notes 
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Two competing note-taking strategies seem to exist for taking notes during Web-based research. Some 
students don’t print very many Web sites, and when they do, are extremely selective about which pages to print, 
making sure the printed pages contain content to answer their question (Fidel, et al., 1999; Jacobson & Ignacio, 

1997). One can conclude that this selective note -taking strategy is particularly effective in environments with 
limited resources (e.g., minimal printing capabilities). 

Other students tend to print many Web pages or copy and paste large chunks of text, rather than take 
selective notes (Eagleton & Guinee, 2003). During this strategy, students make initial high-level decisions about 
whether information is worth returning to later, and then gather it for later examination (Hirsh, 1999). This “wide 
net” strategy seems to be adaptive in situations where Internet access is limited. It allows students to gather large 
amounts of potentially relevant information, which they can peruse offline at their leisure . 

Synthesize Information 

Unfortunately, the majority of students tend not to synthesize the information they find on the Internet 
before presenting it in their research products (Bowler, Large, & Rejskind, 2001; Eagleton, Guinee, & Langlais, 
2003; Gibson & Mazur, 2001). Students report that they don’t copy from the Web or that they are less likely to 
copy from the Web than from traditional print sources (Guinee, 2004; Large & Beheshti, 2000). However, in 
practice, a fair number of students copy and submit content directly from the Web (Eagleton, Guinee, & Langlais, 
2003; Guinee, 2004). Other students make slight modifications to the text they acquire from the Web, substituting, 
deleting, or adding individual words (Guinee, 2004; Large & Beheshti, 2000). 

Cite Sources 

Students tend not to cite Web sources (Bowler, Large, & Rejskind, 2001), even when instructed to do so 
(Guinee, 2004). This finding is understandable, considering it is difficult for students to keep track of the sources 
they are using as they rapidly jump from site to site on the Web. In order to accurately collect their sources, students 
must think of collecting each URL at the precise moment they are gathering information from that site. 

Produce Research Product 

Students consider “research’’ to mean finding information, thus they omit the writing and presenting stage 
of the process (Bilal, 2002; Gibson & Mazur, 2001; Wallace, et al., 2000). As a result, their final research products 
don’t necessarily meet educators’ expectations. As mentioned earlier, some students present material acquired 
directly from the Web (Eagleton, Guinee, & Langlais, 2003). Other students with superb literacy or technology 
skills create flashy research products containing little content (Eagleton & Guinee, 2004). Fortunately, a few 
students successfully navigate the complex process of Internet research and use their found content to generate 
original products. 

Learning to Conduct Internet Research 

In general, today’s students are teaching themselves how to use the Web or are learning about it from their 
peers (Lubans, 1999; Vansickle, 2002). Even in a classroom environment, students ask lots of questions of one 
another and give each other advice while searching (Fidel, et al., 1999). Students say they would often prefer to ask 
a teacher or librarian for help, but these “experts’’ aren’t always available (Fidel, et al., 1999; Lorenzen, 2001). The 
danger in having students teach one another is that they can (and do) propagate misinformation. 

Because teens are learning their Internet skills from one another, they’re also comparing their level of 
proficiency against one another. As a result, the majority of students are content with their searching skills, 
considering themselves to be “intermediate’’ level searchers (Fidel, et al., 1999; Vansickle, 2002). Students feel they 
can successfully search on their own, but will readily ask for help when they run into a problem (Vansickle, 2002). 
When asked if they want to learn more about searching, students respond that they already know everything they 
need to know (Fidel, et al., 1999). 

However, experts disagree. Many researchers feel that Internet searching and evaluation needs to be taught 
explicitly to students (Eagleton & Guinee, 2004; Jacobson & Ignacio, 1997; Schachter, Chung, & Dorr, 1998; 
Wallace, et al., 2000). Students say they would rather learn about the Web in an informal, independent manner, 
such as from individual assistance or posted tips, rather than in a group setting like a class (Vansickle, 2002). This 
preference can be readily accommodated and therefore, educators should consider individual learning approaches 
when designing Internet instruction. 
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Conclusion 

Conducting research is an extremely complex process and performing it using the Internet simply adds to 
the complexity. Students’ approaches to Internet research vary greatly, but most are able to complete Web-based 
research tasks. However, the quality of most students’ performances can be greatly improved. To achieve this, 
methods and materials should be developed to scaffold students through every phase of the Internet research process 
outlined in the presented model. 

Many factors mediate students’ performance when conducting Web-based research. Helping students to 
develop underlying skills, such as computer experience, can facilitate their Internet research practices. Educators 
should also help students develop background knowledge in their topic space and an accurate mental model of the 
Internet information space. Most importantly, strengthening students’ traditional literacy skills, particularly their 
vocabulary, comprehension of expository text, and note -taking, should have considerable positive impacts on their 
research practices. 

Finally, students need opportunities to practice the strategies they are learning and to engage in authentic 
Internet inquiry research projects. This can be achieved by systematically incorporating Internet inquiry throughout 
the curriculum. We must help students view Internet research as a process, not simply an event. 
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Introduction 

The insurgence of technology in educational settings has sparked considerable confusion and controversy 
over the definition of learning objects and how to best to utilize them in an instructional environment. The basic 
conflict stems from the perceived function of a learning object; is it better to learn from or with the object? When 
students learn from technology, they passively acquire knowledge from presented information; when they learn with 
technology, students actively use the learning object. Proponents of learning from technology would advocate using 
television shows such as Sesame Street and Between the Lions to help young readers develop readiness skills. David 
Jonassen, a proponent of learning with technology, proposed the idea of utilizing technology as mindtools to assist 
students in expanding their cognitive capabilities. After analyzing these varied perspectives, it is clear that both 
methods have educational merit and that digital media can play a dynamic role in unifying these diverse s chools of 
thought. Until recently, the definition of the term digital media has been so encompassing that, quite often, digital 
media producers found themselves creating solutions in search of problems to solve. Contrasting that is the fact that 
there are those who characterize learning objects and mindtools too narrowly and the result is that all parties are 
missing an unparalleled opportunity to re -look at these paradigms in an effort to bring them together into a unified 
framework and discover solutions to problems that already exist. 

Learning Objects Defined 

The term learning object originates from another term generally associated with computer programming, 
object-oriented programming (OOP). In object-oriented programming, an object is defined as an object is a unit of 
code that is eventually derived from the process of designing code in such a way that each unit both performs a 
function in the code and can stand alone to become an instance of a particular class or subclass of methods, 
procedures, or data variables (Montlick, 1999). Watson (2001) stated that learning objects were reusable objects that 
were designed for a specific purpose to facilitate learning and could be categorized by using metadata. In other 
words data about data. This categorization enables users to search for, access, and reuse objects as needed. 

In their paper Learning Objects , Todorova and Petrova (2003) stated there are multiple definitions of 
learning objects. L’ Allier (1997) stated, “A learning object is defined as the smallest independent structural 
experience that contains an objective, a learning activity and an assessment” (p 2). In a broader scope, anything can 
be an object (i.e., human being, buildings and even items like buttons, icons and scroll bars), as long as they 
demonstrate certain characteristics. In data modeling, objects are defined and relationships between them are 
established. To be considered an object in programming, that unit of code must meet fairly strict definitional 
standards, such as being sharable and reusable models. Because they are reusable they can be reconstituted to run in 
most circumstances. Like interchangeable network connections, new objects are easily defined without the need to 
know the logic needed to run them, as long as these common gateways are defined. 

Over time, however, new paradigms and broader definitions for the use and definition of objects have 
emerged. They have become part of more complex and multifaceted technological constructs and incorporated into 
new disciplines such as information technology, educational psychology, and instructional technology. In the 
educational realm, objects have also been referred to as instructional objects, educational objects, intelligent objects, 
and/or data objects (Gibbons, Nelson, and Richard, 2000). 

Like their programming counterparts, the most common definition for learning objects indicates that they 
are also model-based, modular, interactive in nature, and can serve various instructional needs. They can be 
effective solutions in many instructional domains such as problem-based learning, functional analyses, coaching and 
feedback, logic, and many of the so-called constructivist approaches (i.e., student learning management, recording 
of responses and selections, etc.). Like those defined in the computer sciences, to be considered an object, they must 
be generative, scalable, adaptive, and more. 

The adoption of learning objects in the educational community has not been without controversy or 
misunderstanding. Educators have disagreed, for example, on whether a learning object need be a finished product 
like a video clip or whether any tool used in the creation of an instructional unit might be considered a learning 
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object. The debate essentially boils down to differences between the way educators understand and differentiate the 
two terms informing versus instructing. The former is a necessary but insufficient and not a complete condition to 
learning. Generally it is accepted that, to be an effective instructional tool, an artifact must also foster the actual 
transfer of knowledge (or cause a change of action or effect attitude) in a structured way that somehow resembles an 
organized taxonomy like Bloom’s. This is the exact rationale many use to argue that the World Wide Web itself is a 
classic non-example of a learning object. While many educators promote the Web as a learning tool that fits many of 
the criteria of learning objects (certainly the Web is reusable and scalable), others believe there are serious questions 
as to whether it fully provides the requisite design concepts, architectures, and tools that make it a valid instructional 
channel rather than being merely a reporting mechanism (Fairweather & Gibbons, 2000). Another non-example 
often held up as a role model for learning objects is the computer itself. While computers can generally free students 
to adjust and tailor similar tasks to their individual needs, without some type of mediated structure, they mostly 
generate random activity without attaining any meaningful instructional benefit. It has been the role of the digital 
programs (i.e., media) to turn the computer into a learning object. 

Lack of a Precise Definition 

Much of the misunderstanding concerning learning objects can be traced to a specific learning theory that 
the particular researcher subscribes to and has based their definition or analysis. Further complicating things is that 
learning objects are often incorporated into general instructional design domains that view them as finished products 
or specific sub-sets of an instructional lesson, rather than a tool for learning. In an attempt to sort things out, Merrill 
(1999) classified four types of "knowledge’ (i.e., learning) objects. He defined entities as devices, persons, places, 
and symbols; properties as quantitative or qualitative attributes of entities; activities as actions the learner can take 
and, processes as events that occur and cause change. Merrill (1999) defined the attributes of learning objects as the 
medium used to deliver them. In this view, the same knowledge objects can be theoretically reconfigured to 
construct different types of ways to inform, encourage practice and rote skills, and/or guide learners. While these 
classifications appear to be outside the bounds of the original definitions and their uses they were intended for 
objects in the OOP domain, they make sense and are clearly definable. Instructional technologists have never been 
able to describe learning as precisely as computer scientists have been able to with programming statements, due to 
the various ways individuals learn. Clearly a case for a more open-ended definition needs to be made. 

Another problem with settling on a more precise definition for learning objects is that many feel that the 
role most often referred to in the analogies used to describe them does not quite fit the precise parameters originally 
used by programmers to describe object models in the computer science domain. Such products like Lincoln Logs™ , 
Erector Sets™, and Legos m readily come to mind. The confusion is exacerbated by the fact that these toys fulfill 
only some of the criteria associated with learning objects as per their more precise definitions: 

• They all contain units or objects of the smallest, fundamental size possible to be of use. 

• They can be assembled into literally any shape and size, and used for a multitude of functions. 

• Their use is very flexible in that some learners or instructors can use partially pre -assembled units 
made up from smaller pieces of these core units and immediately put them to use. 

Upon further review, these toys are more like programming objects than some would like to admit. For 
example, each has elements that may be used and applied and/or built into various new structures without 
decomposing the individual units so they can be reapplied or reused later. Others can assemble structures completely 
on their own from scratch without the use of any pre -fabricated portions. Furthermore, they fit well into the 
instructional domain because most of them can be totally self-sufficient in their use, even though some may need 
instruction and guidance on how to assemble these parts into final products. The fact that they are malleable speaks 
to the fact that they can also be used as tools to spark cognitive development. The fact that they can be used in their 
pre -fabricated form speaks to the fact that they better fit Merrill’ s broader definition and scope of learning objects, 
but might not by those who would view learning objects narrowly as finished units or snippets of media that are 
incorporated into a larger whole. 

Some terms that have come from the literature are instructional object, educational object, learning object, 
knowledge object, intelligent object, or data object, however, the long range analysis of learning objects orthe 
purpose is to facilitate learning. For this article, learning objects will be defined as instructional objects that are 
multifaceted, multifunctional, reusable tools that take on many different shapes, constructs, and context. They are 
technology objects of learning that can assist with the learning process. In the context of this study, students learn 
with the technology not/romthe technologies. 
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Literacy and Learning in the 21 st Century 

As we enter the 21 st century, literacy remains the most fundamental aspect of education. To develop 
literacy, educators must infuse literacy instruction throughout the education process so students can become better 
readers, writers, and content learners. Using scientifically based research, five essential elements for developing 
good readers have been identified. They include phonics, phonemic awareness, vocabulary, fluency, and 
comprehension (U.S. Department of Education, n.d.). According to President Bush (2001), “...the most basic 
educational skill, and the most basic obligation of any school is to teach reading” (f 3) However, in the 21 st century, 
reading is by no means a complete definition of literacy. 

From the simplest time of learning what educators called the “Three R’s” to the more inclusive definition 
of today, literacy has always been a vital part of education curricula (Serim, 2003). In its most basic form, literacy 
can be described as the ability to read, write, listen, and speak as well as critically analyze and express ideas using a 
variety of media or learning objects. However, if we are to define literacy in the context of students’ being able to 
thrive in today’s digital age, we must expand that definition (NCREL & Metri Group, 2003). In addition to basic 
literacy, students are expected to attain proficiency in scientific, economic, technological, visual, information, 
media, and multicultural literacy (NCREL & Metri Group, 2003) so that they are able to be productive citizens in a 
technology rich 21 st century. 

North Central Regional Educational Laboratory and the Metiri Group (2003) asked, “Are your graduates 
ready to thrive in today’s Digital Age? Upon serious reflection, most schools must answer with a resounding. No!” 

(p 4). As the CEO Forum on Education and Technology (2001) concluded in Key Building Blocks for Student 
Achievement in the 21st Century, the definition of student achievement must be broadened to include the 21st 
century skills. Since this report came out, two major initiatives have conducted extensive research and published 
major reports in 2003: the enGauge 21st Century Skills: Literacy in the Digital Age from the North Central Regional 
Educational Laboratory and Learning for the 21 st Century from the Partnership for 21 st Century Skills (North 
Central Regional Educational Laboratory, 2003; Partnership for 21 st Century Skills, 2003). In essence, educators, 
businesses, and industries partnered to develop a model of learning that would help public education systems assist 
students in gaining the skills needed to succeed in the 21 st century, often referred to as the digital age or media age. 
They came up with six elements called 21 st century skills. The first two elements emphasize core subjects and 
learning skills. These elements focus on improving learning by infusing information, communication, critical 
thinking, and problem solving skills within existing school curriculum. The latter categories incorporate 21 st century 
tools within learning skills, context, content, and assessment. The model encourages educators to integrate today’s 
technology with real-world situations so students can develop needed skills with practical applications. (Partnership 
for 21st Century Skills, 2003). Most importantly, the reports stress that we must bridge the gap between the 
knowledge and skills most students learn in school, the way those skills are acquired, and the knowledge and skills 
that they need in 21 st century communities and workplaces. 

The youth of today are inundated with technology that has the potential to extend literacy and allow them 
to actively participate with a variety of media (NCREL & Metri Group, 2003; Serim, 2003). Sixty -five percent of 
children in the United States are already online and the U.S. Department of Commerce estimates the current growth 
rate for Internet use at 2 million new users per month; the majority of which are children and teens (NCREL & Metri 
Group, 2003). 

The world in which our students live is significantly different from the past. Today’s students use cell 
phones, pagers, instant messaging, PDAs, and laptops to connect to friends, family, and others in their community 
and all over the world. Our students now have at their fingertips a digital virtual world- with all its promises and 
pitfalls. Technology can be a valuable tool to achieve instructional objectives if integrated into the curriculum 
appropriately. When combined with the other key factors that increase literacy and student achievement, such as 
clear and measurable objectives, learning objects that increase knowledge, increased time on task, frequent feedback 
and teacher subject matter expertise; technology can help deliver significant and positive results. 

Technology as Effective Learning Objects 

There is considerable evidence that children are born with right-brained cognition, which is aided by media, 
while left-brained cognition has to be developed with the aid of text (Doman, 1984; Shihida, 1994). In order to 
effectively function in the world, students must learn to balance right and left-brained cognition. Robert Doman 
(1984) has suggested that the most effective way to create this balance is to teach to strengths and remediate 
weaknesses. Students that lack appropriate literacy skills will not develop these skills just because they are given 
another book to read, especially if text -based commu nication is a weakness. 



215 



Increasingly, educators are beginning to utilize methods of instruction that include the student’s right- 
brained strengths by integrating learning objects into the traditional curriculum. As stated earlier, learning objects 
are considered instructional objects that are multifaceted, multifunctional, reusable technology objects of learning 
that can assist with the learning process where students learn with the learning objects. The term learning objects 
may encompass a large array of digital resources from digital images to entire Web pages. Computers, videos, 

DVDs, the Internet and television are popular learning objects that are changing the dynamics of learning and by 
combining moving pictures and audio have the ability to appeal to a variety of learning styles 

The success that these learning objects will have in developing 21 st century literacy is largely dependent 
upon the method of integration employed by the teacher. According to Dr. Thomas Reeves (1998), there are two 
distinct styles used to integrate media into the classroom; students can learn with technology or from technology. 
When students learn from technology, they passively acquire knowledge from presented information. Children are 
exposed to the media; then, it is assumed to that they have gained the desired knowledge because they can respond 
appropriately. In order for students to use technology as learning objects that they can learn from, the technology 
must stimulate critical thinking and promote higher order learning skills. The students must use the media tools to 
access, analyze, interpret, and present their constructed knowledge (Reeves, 1998). 

Recent improvements in digital media, digital video in particular, have changed the entire learning 
landscape. We are rapidly moving from a time where students learn from media to an era in which they are highly 
motivated to learn with media. The job of educators is to create an environment conducive to learning from a variety 
of media by ensuring that the employed media correlates with the curriculum and that students are active users of the 
media (Bransford, Klee, Michael, & Warren, 1993). 

Video in the Classroom 

According to John Keller’s (1983) ARCS motivational model, if you gain a student’s attention, make the 
concept relevant, offer a valid challenge, and provide an avenue for success, there is an increased opportunity to 
elicit positive change in the academic success of the student. Today’s generation of students is not strangers to video 
media; everything from their cell phones to their video games uses some type of digital imaging to gain their 
attention. Therefore, it would seem logical that using digital video cameras in the classroom would motivate 
students to engage in the writing and reading process, which in turn could increase literacy. 

That motivational model, coupled with advances in technology and increased knowledge of true integration 
is causing significant transformations in the way students are developing literacy. More educators are starting to use 
video cameras and non-linear editors as instructional tools that help students develop the skills they need to meet 
state standards. Editing products, such as Apple Computer’s iMovie, have transformed the use of video to the point 
where even younger elementary students are capable of creating digital stories. 

Fifth grade students at Sabal Point Elementary in Longwood, Florida use video production projects to gain 
a better understanding of complex concepts while developing reading, writing, listening and speaking skills. During 
the 2004-2005 school years, the 5 lh grade students made several multimedia projects as they explored the concept of 
democracy and the presidential election. Working in three clusters, the students were challenged to create their own 
political parties, develop their own set of relevant issues, choose a presidential and vice presidential candidate and 
then persuade University of Central Florida (UCF) College of Education, educational technology graduate students 
to judge the speeches and issues and then vote for each of the candidates. 

The students from Sabal Point never had any direct contact with the UCF students; their only medium for 
communication was the World Wide Web. The project began with the 5 th grade students reading a book from the 5 th 
grade reading list titled “The Kid Who Ran for President” written by Dan Gutman that meets state standards and was 
an excellent way to get students relating to someone their age running for president. Next, the students had to 
research the complete electoral process. Then, students wrote their campaign speeches, which had to be based on a 
platform with issues which symbolized what their party represented in the campaign. The students also had to write 
campaign slogans, create flyers, and come up with ways to earn campaign funds. The 5 th grade students used digital 
cameras, digital video cameras and iMovie to film and edit videos of their campaign speeches which expressed their 
opinions, newscasts that broadcast their platforms, political advertisements that they used to persuade voters, and 
debates that they hoped would discredit their opponents. The students used Microsoft Publisher and MS Word to 
create a Web page and online newspapers that would help propagate their issues. The interdisciplinary and cross- 
disciplinary curriculum of these lessons created learning opportunities for students to not only learn content and 
meet learning objects but also interact with technologies as they learned the content. 

The same students that did not typically do well reading a textbook or listening to a lecture, flourished in 
the classroom because they were actively engaged in a project that had meaning to them. One student remarked, “I 
don’t mind learning if its fun, I just don’t think learning should seem like work all the time.” Through this project, 
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students were motivated to learn a variety of research, writing, critical thinking, and decision making skills. They 
followed the election coverage using television media, the Internet, and newspapers. Students gathered information, 
analyzed various points of view, and made decisions about how to run their own campaign. Students wrote and 
edited scripts, assigned tasks, worked cooperatively, and managed other students cooperatively. 

The writing process remained the same, but become less tedious when the traditional paper and pencil was 
replaced with a video camera. 

Tablel Comparison of Traditional and Video Writing Process 



1 . Prewriting- brainstorm, gather 
information, take notes, outline 


1. Prewriting- brainstorm, gather 
information, take notes, storyboard 


2. Drafting- put your ideas into 
sentences and paragraphs, make sure 
that there is a logical beginning, 
middle, and end 


2. Drafting- use the video camera to put 
your ideas in a logical format, make 
sure that there is a beginning, 
middle, and end 


3. Revising- read your draft, make sure 
that it has a clear focus, stays on topic, 
and is appropriate for the intended 
audience, add or delete any necessary 
parts 


3. Revising- view your footage, make 
sure that there is a clear focus, a 
consistent topic, and that it is 
appropriate for the intended 
audience, delete any unnecessary 
footage, shot any necessary 
additional footage 


4. Editing- check for grammar, sentence 
structure and spelling 


4. Editing- put clips in order, add 
transitions and necessary audio 


5. Publishing- prepare a final draft 


5. Publishing- create a QuickTime video, 
upload to the Internet 



These conceptual frameworks are not original; however, interactive media is clearly the language of the 
students of today. Paula Monsef (2003) in the Digital Divide Network stated, “Students who may not take to 
learning by reading a textbook or listening to a lecture often jump at the chance to understand complex concepts by 
presenting finished products in the form of a film or a Web documentary or a PowerPoint presentation.” We are 
already seeing this in the county school districts around UCF and all around the country. 

Don Henderson and Marco Torres, both Apple Scholars, have been working with students with projects just 
like these for quite some time. They have expanded these concepts beyond reading and writing to other subjects, 
including curriculum areas important to social studies, applied mathematics, and physics. Similar projects have been 
showing up in other cities like San Antonio, Texas, and Orlando, Florida. In a suburb of Orlando, ESTEEM is a 
local volunteer organization providing neighborhood based programs to children and their families that focus on 
literacy, academic and employability skills training. They have developed a project called Picture It, Write It'. This 
program starts children out with simple storyboards of photographs and/or drawings and challenges children to write 
about them. The program evolves into using video footage shot by the participants that allows the participants to 
develop more complex storylines. Another program underway in local K-12 schools is called UR the Director, 
where students are taught how to read books for content by teaching them the text -to-screen essentials of film script 
writing so they can eventually make movie trailers from the books they are reading. These are just a few of the 
programs that are beginning to develop showing that digital media can make a difference in the learning process. 

Summary 

The goal of the video in the classroom video project at Sabal Point Elementary was to increase student 
achievement with the technologies by making learning come alive to students, have learners actively engaged in the 
process of learning through authentic learning experiences, and make that learning have meaning to their lives 
through the process of learning with the technologies. Students are researched, wrote, created scripts, newsletters, 
flyers, and a variety of other projects that far exceeded the typical paper and traditional classroom assignments. Fifth 
grade students created group video literacy projects for book reports, point of view story retellings, and election 
coverage. Their latest literacy project was a community service documentary; students recorded and scripted their 
efforts to assist the victims of the 2004 Florida hurricanes. Their teacher, Ms. Brandi Evans, stated the students' 
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literacy skills have improved. Students are motivated to read more and complete assignments when the outcome is 
doing something that they enjoy instead of another test. 

Students learning with technologies, such as digital video, incorporate the most important aspects of the 
language arts curriculum; reading, writing, listening and speaking into every assignment all while developing 
literacy and using a familiar medium. In order to write scripts, students have to retell events and facts in a logical 
and sequential manner in addition to summarizing and synthesizing facts from various sources; the ability to 
complete this task is a key indicator of comprehension. Students also gain fluency as they begin to perform their 
scripts and they learn self -correct skills because they see and hear mistakes that they might not have caught if they 
read their scripts alone. 

Teachers also benefit from using video in their curriculum because they are no longer assessing students' 
recall of obscure facts. Video is an opportunity for teachers to conduct authentic assessment of the students' critical 
thinking skills. The teacher has the ability to monitor the students as they research and see where students' 
misconceptions are as they begin to write and perform; the teacher can then clarify or remediate immediately. 

Although the concept of how learning objects should be integrated into educational curricula still remains a 
source of controversy, it is obvious that digital media is the language of this generation of learners. By teaching 
students to learn with technologies, as opposed to learning exclusively from technology, educators are helping to 
equip students with the 21st Century literacy skills that they need. 
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Implementing a Successful Laptop Program 



Thomas E. Haynes 
Culver Academies 
Culver, IN 

As straightforward as it sounds, the three essential requirements to implement a good laptop program are to 

1. Plan well, 

2. Support the plan, and 

3. Realize that the program is dynamic and will require ongoing planning and support. 

The details of a successful implementation have to address the three essential requirements, and these 
details are the topic of this article. The observations are based on the author’s experience with a laptop program at 
The Culver Academies in Culver, Indiana. 



Background 

In 1999 Culver had a Novell network with email provided to faculty and administrative staff by a Microsoft 
server. The school web page was handled by another Microsoft server. All the servers were managed in-house. 

There were several computer labs on campus. Students could do research and get on the internet in the library. There 
was a lab in the Math department for computer science classes. 

The network design was not bridged or segmented, and it consisted of a fiber backbone with cat 5 to the 
academic and administrative buildings. A new Math and Science building was in the planning stages, and it would 
be added to the fiber backbone. The plans for the new building called for a switch in each room and plugs for 
Ethernet and electricity in the floor at each desk. 

Culver issued about 150 laptops in 2000 to faculty and residential/administrative staff. The next year, 
another 750 or so were issued to students. It was difficult for Culver to anticipate the level of network use, and the 
best advice is to predict the absolute maximum possible and then double those numbers. The additional 750 users 
might seem to multiply machines and load in a predictable way, but faculty use is not a good predictor of student 
use. 

The move from a mixed Mac/Windows network to a straight Windows network had been made in the 90’ s, 
and Dell laptops were selected for the faculty machines. These machines had a 500 MHz processor, a 4.2 gig hard 
drive and the memory was upgraded to 128 mb. The machines ran Windows ’98 and Microsoft Office 2000. The 
machines were purchased by the school, and there was no charge to faculty or staff for their use. The plan was to use 
these laptops for five years, and have the students on a four year cycle of replacement with leased machines. 

The intent was to have classes for faculty and familiarize them with the new technology, and this is a 
familiar strategy for schools who are investing in a laptop program. There were a series of classes on the Microsoft 
Office Suite and productivity tools, and the classes were designed and taught by teachers and technology staff. The 
FrontPage class had teachers create a live web page on the web server, and the classes were intended to support 
instructional use. The classes were attended because they were required, but there were few other requirements in 
the process. Teachers were not required to turn in a technology plan incorporating technology for example. 

The staggered roll-out was an advantage in many ways. The teachers were allowed to familiarize 
themselves with the new laptops. The technology staff had a year to anticipate increased support needs, and 
increased loads on the infrastructure could be determined. Instructionally, there was little more use of technology 
during this year, and only about 15% of the web pages were maintained and current six months after the classes. 
Instructional benefits of the year were indirect. 

Technology leaders were identified in each department, and they formed a group called the 
“TechEducators.” This group met with the Academic Dean and the Instructional Technology Coordinator weekly to 
help direct the planning and implementation. Members of this group were enthusiastic, but had a range of skill 
levels. A similar group was planned for students to provide some limited user support in the dorms and in 
classrooms. 

The school had an acceptable use policy in place, and upgraded the Exchange server and increased network 
storage to accommodate the increased number of users. Faculty and staff had no quotas or limits on the number or 
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size of files stored, although they would get a polite email if they were using space inefficiently. This informal limit 
was about 500 mb on the mail server and 3 to 5 gig on the file server. The faculty laptops did not have much room 
for storage, and so the restrictions on network storage were minimal. 

Part of the administrative plan was to make the laptop program as non-threatening as possible for the end 
user. There were support resources , hardware, and classes in place. There were few explicit expectations, although a 
lot of time and energy was being spent on planning for the laptop program. Even the most casual observer would 
appreciate that the program was important to the same people who do performance evaluations and sign contracts. 

The student laptops were leased and prepared for the students in the summer of 2001. Dell laptops were 
selected after a long evaluation process, although an IBM model may have had slightly more support fro m the 
technology staff. Students and faculty were part of the evaluation process. These machines were 800 MHz with 10 
gig hard drives and 128 mb of ram. They were loaded with Windows ’98 and Microsoft Office 2000. There was no 
technology fee associated with the laptop program, and it was paid for out of operating funds and income from the 
endowment. 

The TechEducators were given a reduction in teaching load from four to three classes, and there was a 
budget for instructional software. Wireless hubs were installed to provide students with Ethernet connection, and the 
dormitories were wired for cat 5 and also received wireless hubs. The effectiveness of the wireless hubs took 
everyone by surprise, and plans for the new Math and Science building were modified to include a wireless hub in 
each classroom as well. 

Five members of the tech staff earned a Dell certification so that the school could do repair work and be 
reimbursed by Dell for labor. This was a very good move because it helped to minimize turnaround time on repairs, 
and it kept costs down. The lease for the laptops included 30 extras to provide as loaners for people with laptops 
being repaired. 



The First Three Years 

As mentioned faculty usage is not a good predictor of student usage. Faculty members were using the 
laptops for primarily administrative purposes. They were keeping class records and checking email. Few faculty 
members played networked games like Doom or Counterstrike. Many of the students played games, and the 
broadcast traffic created problems for the flat network design. Peer to peer file sharing had not even been an issue 
for the faculty, and its widespread use among the students raised ethical, legal, and performance issues. Restricting 
access to inappropriate content while permitting a wellness class to research “breast cancer” was a challenge as well. 

Another issue dealt with the implementation of the laptops in the curriculum. High levels of use do not 
guarantee meaningful instructional use, and once users get past the novelty, the real goal is to communicate 
instructional content in a deeper, more effective, or more efficient way. The TechEducator group worked with the 
ISTE standards to understand them and to adapt them for our environment. The standards were then communicated 
to departments and feedback was incorporated in the final version. This process helped keep the instructional 
purpose of the laptops a topic of conversation, and it made departments and the department chairs part of the 
process. 

In the third year with the student laptops, several things happened. The campus network was almost shut 
down by the MSBlast worm as students returned from summer break, and it became obvious that the network 
needed to be restructured by switches into subnets to restrict the broadcast traffic and improve performance. The 
student file server was moved to a Linux machine that provided “user directory” web pages for the students. 
Teachers (especially the younger ones) were increasingly making non-trivial use of the technology in their 
instruction. Technology integration was no longer the point of a lesson, but a means to an instructional goal. 

The student web pages were contained in a folder mapped when the students logged in. They can store 
project files there and the Information Literacy classes included a section on working with web pages. This 
functionality is very powerful, and it was easy to set up. Few teachers have made use of the feature, although it has 
been an excellent resource for about 10% of the faculty. The cost on this was minimal in that is was installed on a 
“retired” server and there were no licensing costs associated with the operating system or the software. This server 
and the student projects were the topic of a presentation at the National Council of Teachers of Mathematics 
(NCTM) national conference in Philadelphia - 2004 (Haynes, T. (2004, April). Digital Portfolios in the Math 
Classroom. Retrieved October 8, 2004, from http://academies.culver.org/nctm04/indexp.html ). 

Another excellent tool that has been very popular with faculty and students has been the program Moodle. 
This program is a content management system that allows for online quizzing, journals, and threaded discussions in 
forums. It is als o open source and there are no licensing costs or fees. This program is home to about 25 active 
“courses” used by 300+ self-registered students, and it has proven to be a valuable resource for submission of 
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assignments and peer assessment ( The Culver Academies, (n.d.). Retrieved October 8, 2004, from 
http://academies.culver.org/moodle ). 



The Present 

The faculty laptops have been sold back to Dell, and for the 2004 - 2005 academic year, the student lease 
was converted to a new model. The teachers and students now have the same laptop with slightly different software 
packages. The operating system is now Windows XP and the office suite is Microsoft Office 2003. The new laptops 
have a much longer battery life and are much faster. They have 40 gig hard drives as well, and there is less 
dependence on the network storage. The network has moved to a Windows Active Directory domain, and the 
centralized password database has eased management. Moodle and the Linux student file/web server both 
authenticate against the domain. 

Some of the faculty members who have a technical inclination have discovered that they can distinguish 
themselves with good effective use of the laptops in the classroom. There has been good support for resources 
although the instructional technology budget has been trimmed where possible in the past two years. A characteristic 
of Culver is its excellent funding of professional development, and many teachers have taken advantage of 
conferences and workshops. This professional development has increased the pace of integration. 

The Dean of Faculty has asked faculty members going through a five-year review to submit their portfolios 
digitally. The cohort going through that process is very aware that it is much easier to organize digital materials in a 
portfolio than it is to convert them to digital format. This portfolio process has helped to focus the attention of those 
preparing for that process. There is efficiency in being able to develop materials for students that can then be used to 
support performance in an evaluation process. 

At this point even the most cynical faculty members no longer see the laptops as a phase the school will 
move through. The laptops are seen as tools, and changes have been made in the acceptable use policy and 
classroom policies to better address the realities of the learning and living environments. 

One of the issues currently is that the laptop program has been taken for granted in some ways. The 
TechEducator group has recently recommitted itself to a standards -based analysis of instruction and development. 
The modified ISTE standards have been brought out again, and the group has realized that they may have been 
distracted by details and were not attending to their primary task. 

As technology use has increased, there have been requests for hardware like projectors. These requests 
demonstrate the increased use of the technology resources, but a concern of the TechEducator group is that a 
teacher-centered classroom with technology is still a teacher-centered classroom. The process of curriculum design 
requires an analysis of goals and the role of technology in best achieving those goals (Brewer, R. and Moore, J. 
(2004). Curriculum Design and Technology Integration. Learning and Leading with Technology, 32 (1), 42 - 45.) 

Problems 

It would be disingenuous to document the success of the laptop initiative without an honest appraisal of the 
problems. There have been temporary technical problems, but ongoing challenges fall into two categories: financial, 
and social. 

The damage to the laptops makes the Dell Complete Care well worth the cost. Broken laptop screens and 
keyboards missing keys add up quickly. Accountability is a problem, and how best to charge the students for repairs 
in a boarding school is a problem. If there is no incentive for students to take care of the laptops, they are unlikely to 
take good care of them. At Culver, students are reissued the same laptop every year, and this improves care for 
some. This problem is still unsolved. 

Some teachers are not interested, not willing, or not able to learn how to use the technology. These teachers 
are usually over 50, and they are often considered “very good” teachers. With the addition of the laptop layer, they 
are understood to be “good” traditional teachers but are not outstanding. They observe that the environment has 
changed and blame technology for having changed the environment. 

Since these teachers may be in positions of leadership, they are likely to slow the process of integration. It 
is difficult to know what to do with these slow or unwilling adopters. One approach would be to include technology 
goals in performance evaluations and move these teachers out of leadership positions. Tenure or similar policies 
may make it difficult to remove them, but negative influences have to be minimized. Another “kinder, gentler” 
approach might be to reward the most effective teachers and wait for the less flexible resistance to buy in or retire. 

A laptop program is expensive, and there are those who may be willing to question the return on 
investment. There are many incentives for a laptop program, and it makes good press. This quality helps recruit 
faculty and students. The motivation and the measuring stick for the program should be instructional goals, however. 
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Either the technology helps the learning/teaching process or it does not. As with any change in instruction, be 
careful not to gauge effectiveness too early. There is a period of transition and until the novelty wears off, there may 
be trivial instructional use. 



The Future 

It is tough to predict the future, but a commitment to sound curricular design and a goal-driven student- 
centered learning environment is an important part of a successful school. Technology is one of the ways that work 
is done, and if it can increase efficiency or make instruction more effective, it has served its task. The laptop 
initiative at Culver has provided many opportunities for teachers and students. The laptops have changed the way 
people communicate and collaborate, and technology integration is increasingly important to teaching and learning. 
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E-Listening: Transforming Education Using Collaborative Tools for 

Assessment and Evaluation 

Mari M. Heltne 
University of St. Thomas 

Judith B. Nye 
Luther College 

What People Really Care About 

When the AAHE (American Association for Higher Education) Assessment Forum developed the 9 
Principles of Good Practice for Assessing Student Learning, Principle 7 stated that assessment makes a difference 
when it illuminates questions that people really care about” (Astin). The first principle recognizes that 
assessment “is not an end in itself but a vehicle for educational improvement.” This paper discusses how our 
department curricula, classroom environments, and ultimately student learning might be improved by “high tech” 
assessment and evaluation techniques to find out “what people really care about”. Collaborative technologies are 
used to gather and process the opinions of students, faculty, and other stakeholders. 

Five years ago, the college president challenged the faculty to examine how students learn and what each 
department’s curriculum should look like for the 21 st Century. He strongly encouraged us to begin active planning 
for the continued intellectual vibrancy of the College and its curriculum. Because “students learn more when there is 
a match between their abilities and the curriculum” (Ratcliff, 1995), to remain intellectually vibrant demands that 
our college examine regularly what we are trying to accomplish as well as what is contributing to student success 
and what is not. Several groups met regularly to discuss curricular changes: Faculty Council, Administrative 
Council, the President’s Council, Teaching Groups, Academic Affairs Committee, and fifty senior students. The use 
of collaborative technologies allowed the following: gathering of divergent opinions at times convenient to 
participants; meetings dominated by content, not personality; adherence to a structured agenda, which resulted in 
consistency in issues discussed without loss of those ideas after the session ends. Throughout the sequence of those 
meetings, there was convergence of the following agreements: we validated a common core set of required courses 
for all students, but called for rethinking its content and staffing; we challenged the structure of the current 
distribution system; and we validated the need for more intentional inclusion of writing throughout the curriculum. 
The outcome of the meetings was a mandate for forming a curriculum review team, which recently finished its 
work. 

Educational institutions nationwide are expected to be increasingly accountable for the attainment of the 
stated goals in the form of demonstrable changes in students. Curriculum is one part of the total institutional 
improvement which, in combination with other qualities, has an important impact on student development. It seems 
imperative, then, to include our students in the process of evaluating the curriculum and the value faculty members 
add to their educational process. Using the collaborative technologies as a means for collecting these opinions 
provides a safe and exciting forum for discussion of issues important to the students. Table 1 summarizes how six 
departments used the collaborative facilities for assessment purposes. 



Education 


To assess the teaching skills needed by majors 


Music 


To assess the information technology needs for music courses 


Computer Science/MIS 


To assess the adequacy of computer lab support for departmental courses 


History 


To assess the adequacy of history curriculum in meeting certification 
requirements for teacher licensure 


Chemistry 


To discuss the changes needed, problems of, teaching of, and the 
contents of the general introductory chemistry course. 


Freshman English (Paideia) program 


To discuss and plan the curriculum for the 16 th Century unit of the course 



E- Listening and Collaborative Technologies 

We define “E-Listening” as the use of collaborative technologies to gather and process the opinions of 
students and other stakeholders for departmental improvement, and the attempt to extend the process and its benefits 



224 




to the broader institution. Collaborative technologies are broadly defined as those that enable collaboration among 
individuals engaged in a common task (Kock, 2000). In this case, the common task is assessment for departmental 
improvement. 

This process of improvement depends on continuous feedback with appropriate response. It begins with the 
collection, organization, analysis and reporting of student opinions and assessment data. The authors were charged 
with designing and implementing our departmental assessment programs. These included all the usual tasks of 
writing of mission statements, identifying goals and objectives, and developing means to assess whether we had 
accomplished what we hoped. We represent two different undergraduate departments (Department of Education and 
Department of Computer Science) which share the requirement that they must solicit, organize, analyze and report 
on the annual collections of graduating senior exit interviews. The tasks are cumulative, in that prior information is 
compared with the most recent collection. The enormous amount of data proves cumbersome to record, analyze and 
store. 

The Software Tools 

Imagine a tool that allows the structuring of assessment or evaluation questions and activities, the capturing 
of important ideas, prioritizing issues and opinions, instant reporting of results, use of an outline tool to write 
planning documents based on the data collected, and tools to create reports on any part or all of the data collected: 
that's what collaboration tools can do! 

We designed and coordinated assessment activities using two different collaborative technologies. Group 
Systems and Facilitate.com. In Figure 1, students are shown in the Round Table Room at Luther College, using 
software tools such as Brainstormer, Topic Commenter, and Categorizer, to offer their opinions during such 
activities as the senior exit interview and course evaluation. Although this electronic meeting room is very 
conducive to providing each participant with unobstructed views of other participants, the facilitator, and the public 
screen, the same software can also be successfully used in an ordinary computerized classroom such as found on 
most college campuses. The software allows everyone to “speak all at once” via the computer. Students type in 
ideas at the same time, and each person sees the input of others, stimulating further thought. 




Figure 1. The Round Table Room with Collaborative Technologies 



Most collaborative software programs offer as the initial tool an Agenda program which provides a 
framework for the assessment or evaluation activities. It prompted us as facilitators to develop a specific plan. 
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specify the exact information needed, and keep track of all participants. Electronic Brainstorming is an idea- 
generating tool that enables students to share their ideas anonymously and simultaneously in response either to the 
specific questions posed to them, or to actually be given the chance to suggest additional questions they find 
important but missing. Tools such as Topic Commenter enable students to easily comment on the questions planned 
in advance by the facilitator of the assessment or evaluation session. The Categorizer program assists in analyzing 
the current information, sorting it into categories, identifying and adding missing ideas, and allowing for more in- 
depth comments on current ideas. Voting tools of most collaborative software allow participants to rate and rank the 
issues presented to them. Most collaborative systems also allow for online surveys. They store the data so that it can 
be automatically arranged into formatted reports, thus allowing for easy access of historical data. Three of these 
tools are explored further. 

The actual screens seen by students during a course evaluation session are shown in Figure 2. Three major 
questions were posed: “What helps you learn in this course?” “What do you think needs improvement?” and “What 
are your specific suggestions for change?” The electronic discussion is anonymous, with the software adding 
random numbers to student comments. This facilitates reacting to or answering another comment on the page. In our 
experience, we have found that professors elect to use this evaluation at midterm so that student perceptions can be 
incorporated into the planning for the rest of the course. 




Figure 2. Topic Commenter tool 

One of the most useful tools in a collaborative system is one that lets students categorize the lists of ideas 
or answers that have been entered. In Figure 3, students were asked, “What helps you learn in this course?” The 
categories into which the answers might fall are shown at the right. This tool is mo st important in eliciting actual 
questions that students wish to pose. With a prompt such as, “What ideas do you wish to pose for discussion of 
departmental advising?” the facilitator can sort the questions into like categories, and transfer them to the discussion 
tools shown in Figure 2. All this is done in a matter of seconds. 

Another favorite tool for evaluation and assessment sessions is the one that allows students to make 
decisions and determine degrees of consensus or conflict. Multiple voting methods are allowed, including Yes/No 
and True/False, Top “n” favorites, customizable point scales, and Likert scales. Figure 4 shows a vote with only 4 
options, but many different scales are possible. Facilitators have the option to allow an odd number of choices, thus 
giving the student the alternative of “middle ground”. Results are immediately available, so students can see if their 
opinions vary a great deal from their peers. Viewing and discussing the results of the vote often leads to further 
questions and revealing comments. 
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GroupSystems - Summative - [Main Points - What Helps You Learn? (Categorizer)] 
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1. What helps you learn in this course? 

2. 1 liked the study groups that were required. 

3. The study guide in the website for the course was really helpful. 

4. Professor was always available during office hours. 

5. 1 felt like I could ask questions when I didn't understand something. 

6. 1 really Iked the fact that the professor didn't stand up and "talk at us" all the time. We had 
to read the text and then we did things in class to make it more understandable. 

7. You could tell that she really liked the subject. She got me excited about it too. 

8. The textbook was good, but the research articles were better and helped me learn more i 

about the research process. \ 
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Figure 3. Categorizer Tool 
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Figure 4. Voting Tool Rating Computer Lab Support 

One of the assessment activities that was most effectively supported with collaborative tools was the Senior 
Exit Interview. The purpose had always been to give students the opportunity to provide feedback to the department 
by providing them the chance to assess us on several dimensions. The Exit Interview includes questions on their 
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perceived academic accomplishments, our advising expertise, and the general support offered by the department 
during their time in college. Figure 5 shows the electronic discussion prepared for the group meeting of senior 
majors in Elementary Education. Information gained from the analysis of Exit Interviews is used to revise the 
curriculum, the teaching methodologies, the advising, and to make it even more accommodating to student needs. 




Figure 5: Topic Commenter Tool with Senior Exit Interview 
Assessing the Assessment Tools 

Classroom evaluation and student assessment just became easier and actually enjoyable by implementing 
the activities through collaborative technologies. Instead of listening to individual students in dozens of hours of 
individual meetings in exit interviews, and instead of endless meetings of faculty where opinions are lost once the 
meeting is over, we now have all the data that was gathered by the software stored in an organized fashion. It also 
became possible to store many years of data, which makes it easily accessible by institutional decision makers. An 
added benefit was the visibility of program improvement efforts to the college community. Students, faculty and 
other college constituents expressed positive feelings about their inclusion in these efforts. 

We find that students are eager to express their opinions in this “E-Listening” environment, and they tell us 
that the setting of an anonymous electronic discussion is very freeing and inviting. Faculty members find it easy to 
access the files, whether stored recently or in years past. The choice of several different types of automatic report 
generation makes structuring the data remarkably easy. 

Another guiding principle of the AAHE Assessment Forum is that “assessment works best when it is 
ongoing, not episodic. Its power is cumulative.” (Astin) Assessment starts “with the questions of decision makers, 
involves them in the gathering and interpreting of data, and informs and helps guide continuous improvement.” 
Collaborative technologies provide the means to collect ongoing, relevant information from all groups of campus 
constituents for the purpose of institutional improvement. The environment helps illuminate questions people care 
about. 
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Abstract 

The purpose of this study is to investigate story telling-retelling as a learning strategy to facilitate 
meaningful learning on environmental education in web-based learning environments. Because story telling is a 
way of making meaning of the individual and social life, it can build a richer context so that learners can enhance 
environmental ethics through indirect experience. For the purpose this study develops a learning environment via 
computer networks, which enables learners to understand the natural world and to build environmental awareness 
with storytelling at the elementary level. It was designed for facilitating narrative inquiry with individual and 
collaborative learning through online activities. From the theoretical and practical review, this study suggests 
design strategies for building a cyber learning environment with story telling. 

Introduction 

There are some aspects of human experience that cannot be revealed with traditional scientific thinking, so- 
called logical exposition. Narrative inquiry has been considered as an alternative mode of thinking and learning. 
Narrative inquiry is a way of understanding, organizing and communicating experience as stories, lived and told. 
Within the inquiry field, we live out stories, tell stories of those experiences, and modify them by retelling and 
reliving them. Because stories form the intellectual and practical nourishment of oral cultures, and to the extent that 
our modern literate culture retains oral practices, narrative continues to play a vital role in teaching and learning 
(McEwan, and Egan, 1995). It is not just a manner of speaking but foundational to learning as a whole. Through 
storytelling individuals can learn to express themselves and make sense of the external world. 

Narrative inquiry, as a way of making sense of a human’s life and world, has been studied in various 
approaches. For example, Clandinin and Connelly (2000) refer to narrative inquiry as a research method to 
understand teachers’ knowledge and to enhance teaching abilities. They believe that experience happens narratively, 
so educational experience should be studied narratively. McEwan and Egan (1995) consider some implications of 
narrative in the practice of teaching and learning. Teachers can use narrative as fundamental to represent contents 
and to communicate with students . Students can use narrative as critical to express their thinking and learning, and 
to explore the connection between the self and the world. Storytelling as a particular form of narrative inquiry will 
bring to the fore special features of individuals ’ thinking that have tended to be somewhat neglected in more 
traditional studies . 

Storytelling has been considered as a mode of experience in a variety of academic areas, such as, arts, 
literature, and even science. When science attempts to understand the story of the universe, storytelling is one 
important way to recognize, interpret, and construct interaction between human and nature, especially for building 
environmental ethics. 

Nowadays many nations institutionalize regulations related to environmental conditions for the 
preservation of the natural environment. Some other efforts have also been made to enlighten people on 
environmental awareness and ethics. Historically environmental education spent an inordinate amount of time 
teaching facts and principles to memorize. This kind of learning is important for learners to understand 
environmental conditions, but if the learning is limited to only the fact learning, they can do little with science (Ellis, 
2002). Beyond this, learners need to know how nature functions, and how it is related to human life. Storytelling can 
be a positive way to achieve this kind of learning. 

Based upon current educational trends, this study developed a cyber learning environment via the Internet 
and the world wide web, that enables learners to understand natural conditions, to articulate concepts and theories, 
and then to build environmental awareness with storytelling methods at the elementary level. 

In this study, the learning environment was designed for individual and collaborative learning through on- 
line activities. Teachers can be on-line tutors to support learning processes, and learners can be story-tellers to foster 
deeper understanding about the environment. 
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Storytelling as a way of experiencing the world 

Narrative is a mode of knowing and understanding that captures the richness and variety of meaning in 
humanity as well as a way of communicating who we are, what we do, how we feel, and why we ought to follow 
some course of action. A narrative involves facts, ideas, theories, and dreams from the perspectives and in the 
context of someone’s life. Individuals think, perceive, interpret, imagine, interact, and make some decisions 
according to the narrative elements and structures. 

The story or storytelling, as a form of narrative inquiry, has received a lot of attention by educational 
theorists and practitioners for a long time. One influential version of this inquiry is represented in educational theory 
of John Dewey, who devoted his life to the study of human experience. According to Dewey, educative experience 
is liberating and uniting, in the sense that it opens the continuous path of reconstructing and recreating the habituated 
meanings of the world as well as the enduring attitudes of the self. A genuinely educative experience must build up 
an individual’s continuous reconstruction, moving from past and present to future experience, and involve the 
tensional transaction between internal conditions of the individual and his social world (Dewey, 1938). Each learner 
must reconstruct the periods, phases, or levels of the growth of human mentality. Based upon Dewey’s thought, 
storytelling, as a fundamental attempt by human beings to experience the world, has temporal context, spatial 
context, and context of other people. According to Bruner, story must construct two landscapes simultaneously - the 
outer landscape of action and the inner one of thought and intention (Bruner, 1991). Narrative is a fundamental 
aspect of meaning construction, which is a negotiated activity that starts in early childhood and characterizes the 
whole of human life (Fusai et al., 2003). Human life is filled with narrative fragments, enacted in and reflected upon 
storied moments of time and space. Narrative thinking is a key form of experience, and a key way of acting upon the 
reality. Storytelling gives individuals chances to understand others’ narrative in a social context, and to clarify their 
own thinking. 

Stories have two functions for learning in the epistemological and transformative view. In the 
epistemological view, stories include a certain kind of knowledge that learners should possess to understand their 
experiences and to fully participate in their social community. In the transformative view, stories are designed to 
provide moral messages taken to heart and to transform a person’s way of life. For the former view, stories can be 
used as exemplars of concepts, principles, or theories, and as cases to represent a real situation or a problematic 
situation needing to be solved (Jonassen and Hernandez-Serrano, 2002). For the latter view, stories should provide 
learners with an opportunity to rethink the given stories and retell them in terms of their interpretation. Through 
retelling or rewriting or creating a story, individuals can enlarge their experience and be involved in mutual 
interdependence and growth (Crick, 2003). To compose their own stories of experience is central of narrative 
inquiry, and a way of enhancing an individual’s experience and social interaction. 

There have been several attempts to apply storytelling in a learning situation. Some have focused on using 
technology to support teachers and learners for storytelling, such as using a word processor to write a story, and 
employing communication tools to share a story among learners (e.g. Fusai et al, 2003). However, there are few 
opportunities to utilize storytelling for the enhancement of learning processes in cyberspace facilitated through the 
internet and the world wide web. Some developments have presented stories as an electronic form using texts, 
graphics, and sounds on a screen for providing learners a certain kind of knowledge. However, some other 
applications should be discussed in terms of instructional design rather than just putting stories in an electronic form 
in that storytelling can be assumed to have more possibilities for learning and teaching. 

Cyberspace, as an alternative living and learning environment for individuals, has been discussed regarding 
its possibilities to expand learning opportunities. However, people sometimes encounter harmful and useless 
information rather than the well-qualified, and work in a sequential thinking process without genuine reflection. To 
counter the undesirable aspects of the learning environment in cyberspace, storytelling can be a good way to 
improve individuals’ experiences in cyberspace in that storytelling gives opportunities to think and rethink about the 
self and the world. Moreover, the features of cyberspace may extend the potential of storytelling in that individuals 
can use computer tools to make their learning processes more efficient, and thus interact in both asynchronous and 
synchronous modes in cyberspace. 

Design Issues to build web based learning environment 

The following assumptions are considered to build a learning environment for narrative inquiry based upon 
the theoretical review. First, storytelling-retelling will provide learners the setting in which to understand reality in a 
social context, and to transform experience in a temporal context. Telling stories can assist individuals in the sharing 
of their human diversity, and mediate in the process of exploring and articulating their identity from a particular 
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perspective. Second, teachers can be facilitators to lead learners’ transformative experience and co-learners to 
cooperate in the learning process. Teacheis are pivotal in shaping the learner’s educational experience. Teachers 
must have more diverse roles in learner-centered learning environments than in teacher-centered environments. 
When students experience someone’s stories teachers must pay attention to their understanding. When students 
engage in their own storytelling teachers must lead their students in the right direction for enhancing future 
experience. Third, cognitive and moral development in environmental education can be enhanced in the process of 
telling and retelling individuals’ experiences. The ultimate value of environmental education goes beyond the 
inculcation of facts or information to environmental ethics including aesthetic appreciation (Carr, 2004). Story 
telling-retelling can be an alternative strategy connecting learners to nature, and promoting their moral and aesthetic 
experience. 



Learning activities 

Based upon the assumptions, two kinds of main learning activities are selected: listening and watching a 
story, and telling and retelling a story. 

Listening and watching a story 

A story, as a form of someone’s narrative, can provide a richer example of how an individual interacts with 
the world in a holistic approach. When learners indirectly experience stories told to them their knowledge and skills 
would be extended at both cognitive and moral levels (Hernandez- Serrano and Jonassen, 2003). 

In the activity of listening and watching a story, one or more stories are presented in an animation style, and 
students articulate what concepts and principles the story tells them. Each story consists of a problematic situation 
related to environmental conditions, characters concerning the situation, and a plot configuring to beginning, middle 
and end phases. Stories were constructed by experts of environmental education, modified into the prescribed 
learning modes by instructional designers, and developed by computer programmers in terms of a team approach. 
Concepts and principles related to environmental awareness are embedded in each story. This activity offers learners 
contexts in which to experience external environments. 

Telling and retelling a story 

When individuals retell or rewrite their own stories beyond listening and watching a story told, many 
opportunities are provided for the enhancement of reflective experience and in mutual interdependence and growth 
(Crick, 2003). To compose their own stories of experience empowers learners to transform individual experience 
and social interaction (Clandinin, and Connelly, 2000; Fusai et ah, 2003). 

In the activity of telling and retelling a story, students construct their own story on the basis of concepts or 
principles extracted from the reviewed story. Then they can get teachers’ advice and peer reviews, and also provide 
peers their own feedback through the sharing opportunities. It can give Individuals the opportunity to cooperate in 
the retelling of a story with their peers and teachers. This activity would provide learners transformative experience 
differentiated from that of the beginning state. 

Learning process 

The learning environment constitutes four phases for supporting the learning activities: telling a story, 
articulating what the story was, and retelling anotherstory including sharing the story with others (see Figure 1). 
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Figure 1: The main steps of the learning process 




• Introduction: the learning objective and learning contexts are explained before getting started. 

• Story telling: the pre -designed animation as a story is presented in the interactive mode. 

• Articulation: the concepts and principles included in the story are articulated and reviewed in text 
and graphic styles. 

• Story retelling: it requires learners to retell or create their own story using writing tools and 
sharing it with others using communication tools. 

The whole process provides steps set up in a recursive, non-linear way, which enables learners to go 
forwards or backwards within the learning process by their choice. 

Support system 

The environment also provides resources and tools that participants can use throughout the learning 
process, such as glossary, experts’ knowledge, and communication tools. Resources include facts, principles and 
theories in forms of glossary and information bank. Resources can support learners to articulate and to extend the 
prescribed knowledge. Two kinds of tools, a writing tool and communication tools are provided to assist in 
expressing learners’ thought and emotion, and in enhancing mutual understanding. Tools enable learners to organize 
and to present their understanding in concrete ways, and to share their experience with others. 

Teachers must have diverse roles to lead learners into genuinely educative exp erience. A teacher can be a 
tutor for supporting learners to understand what stories tell, a model writer for assisting learners to tell their own 
stories, and an evaluator for reviewing and promoting learners’ experience. To be successful in this environment, 
teachers must be knowledgeable in environmental education, and sensitive about learners ’ progress and learning 
contexts. 



Conclusion 

Learners can encounter mis -educative experiences such as decontextualized learning contents and 
disconnected activities with their interests in many classroom learning situations. Human experience in cyberspace 
is also subject to contamination with under-qualified sources, for example useless and harmful information, and a 
sequential thinking process without genuine reflection, even though cyberspace has the potential for positive effects 
on learning. Storytelling-retelling, as a way of narrative inquiry, would be considered to offer learners genuine 
learning experience and to transform the whole life in special and temporal contexts. 

Some field trials with this environment in elementary environmental education have shown some 
interesting results in terms of formative evaluation. First, learners exhibited greater attentiveness to watching the 
given story but revealed some difficulty in retelling the story. This would indicate a need of for strategies to support 
learners benefiting from new experiences in writing. Second, teachers indicated the positive possibilities of the 
learning environments for better contexts of environmental education. 

In the theoretical aspect, this study would inform how narrative can work in the cyber learning 
environment. In the practical aspect, this study would provide some strategies to design story -based learning 
environments. 
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Abstract: 

The current study identify the potential problems of current web-based instruction and learning 
management systems in terms of its lack of flexibility and customization required for individual learners ’ different 
goals , backgrounds , knowledge levels, and learning capabilities. Advanced adaptive learning management system 
technologies are able to give possible solutions and mechanisms for adaptively foreseeing, monitoring, 
organization, and evaluating individual learners within learning management systems. According to Brusilovsky 
and Vassileva (2003) ’s suggestions, the authors are introducing the examples of adaptive system mechanisms for 
adaptive self-regulation can be such as Glossary, Indexed Textbooks, Advanced Navigation, Direct Guidance and 
Adaptive Navigation Support, Adaptive Guidance for Prerequisite-based Help, for large scale of web-based 
learning management systems. 



Introduction 

A lot of organizations, nowadays, implement Learning Management Systems (LMS) or Learning Content 
Management System (LCMS) to manage, track, and quantify all of learning activities and resources in an institution 
to meet the individual and organizational needs as one of the effective advanced learning technology. There are 
some of well-known learning management systems particularly developed for university such as WebCT (Goldberg, 
Salari & Swoboda, 1996), Virtual-U (Harasim, Calvert & Groenboer, 1997), FLAX (Routn, Graves & Ryan, 1997), 
TopClass (WBT Systems, 1997) and Learning Space (Lotus, 1997). They are also commercially available for 
corporate and government settings as well. Some organizations, moreover, invest fortunes to build their own 
LMS/LCMS since they realize the inevitable needs of managing, reusing, and utilizing their valuable electronic 
assets including knowledge, information, and resources. 

Is LMS/LCMS valuable enough for organizations to invest considerable budget for it? LMS/LCMS can be 
defined as electronic management systems, which enable to design, review, converse, store, maintain, analyze, and 
upgrade electronic content. There can be more creative processes of dealing with electronic content depending on 
the users requests. Then, what is electronic content? The formats and modes of electronic content can have various 
shapes and modes. It can be web pages having various elements such as text, graphics, controls, multimedia, and 
etc. At the same time, it can be various applications, database, various types of files, programming logic or naming 
conventions. 

In order to make the electronic assets available for being used for authentic problem solving, the learners 
should go through a series of complicated processes within LMS/LCMS. When we think about the whole process 
and resources of LMS/LCMS in terms of human learning and performance aspects, it involves complicated steps of 
incorporating the abundant resources into authentic problem solving. That is to say, the learners need the techniques 
and strategies to use them uniquely for their knowledge construction, task completion, and performance 
improvement. Otherwise, the fortunes of investment for LMS/LCMS cannot give the expected return on investment 
for their organization. Learners are likely to face with potential problems of managing the breadth and depth of 
electronic resources, driving them as they planned, incorporating them into their performance, and optimizing them 
for their workplace. 

Current Learning Management Systems 

The adaptive features of LMS/LCMS become more necessary since the learners visit their LMS/LCM S to 
meet their own unique needs based on their current readiness. Individual learners have their own experience, 
knowledge and skills, as well as cognitive abilities, personality traits, learning styles, interests, and motivation 
before they approach the electronic contents in LMS/LCMS. The main difficulties of learning management systems 
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come from the variety of course forms and resources. Students have various learning sources such as lectures, 
tutorials, examples, quizzes, and assignment, which are not likely to be organized nor managed to serve individual 
learner performance efficiently. Moreover, in many current Web-based courses, the course material is still 
predominantly oriented for traditional on-campus audiences who have a lot of similar demographic characteristics. 
Web-based courses, however, are to be used by much wider variety of learners than any campus-based courses. 
These learners may have very different goals, backgrounds, knowledge levels, and learning capabilities. The only 
way to enhance it is to make the course material richer and more flexible so that different students can get 
personalized content and a personalized order of presentation. 

Adaptive features should be one of the main functions of LMS/LCMS to enhance learning and performance 
of its target audiences. The systems are required to be more adaptive to individuals so that they can really meet the 
learners’ unique needs just on time. Current Web-based courses in this respect are not “supportive”. Neither the 
teacher nor the delivery system can adapt the course presentation to different students . Some students waste of their 
time learning irrelevant or already known material, and some students fail to understand (just misunderstand) the 
material and consequently overload a distance teacher with multiple questions and requests for additional 
information. Here we can see the needs of adaptive Learning Management Systems for Web-based instruction. 

The rapid progresses of web and computer programming technology have enabled the Web-based adaptive 
functions but the functions have not been delicately designed in the manners how users effectively learn, manage, 
incorporate all the overwhelming electronic contents into their authentic environment. Even though TopClass is 
capable of annotating (Bursilovsky et al., 1998), they do not fully satisfy with providing domain structure, which 
indicates the location of current pages and relationship with other pages. Therefore, the needs of implementing 
adaptive features to self-regulate their process of using LMS/LCMS will raise up the current LMS/LCMS to the next 
level. Then, what kinds of adaptive features and functions will be the most important for the users of LMS/LCMS? 

In this paper, we would like to particularly suggest the adaptive features for automated self-regulation support. The 
adaptive self-regulation features can support and guide users to plan, manage, monitor, and evaluate the whole 
process of implementing LMS/LCMS to meet their customized needs. The following sections will introduce the 
possible self-regulation strategies for planning, managing, monitoring, and evaluating electronic resources, which 
can be incorporated for effective design of adaptive LMS/LCMS. 

Self-regulation Strategy for Adaptive Learning Management Systems 

In order to solve the potential problems in customizing the web-based learning management systems and to 
meet the different individual needs, the self-regulation strategies and its adaptive system technology might be 
effectively implemented as the problem. Self-regulated learning is an important aspect of student academic 
performance and achievement especially in online learning settings, which lack the physical face-to-face interactions 
with instructor and other students. Self- regulation can be broadly defined as the efforts by students to monitor, 
manipulate, and improve their own learning (Corno & Mandinach, 1983). Self -regulation includes factors such as 
resource management, goal setting, success expectations, and deep cognitive involvement (Trawick & Corno, 1995). 
Self-awareness, self-monitoring, and self-evaluation are major issue for successful self-regulation strategies 
(McCombs, 1989). Social cognitive theory, being rooted in Bandura’s theory, has been successfully used to explain 
the function of self-regulation in academic settings (Bembenutty, & Karabenick, 2004). According to social 
cognitive theory, successful students are those who actively engage in self-regulation of their motivation, cognition, 
environment, and behavior (Zimmerman, 2000). Although there are many ways to define and to articulate learning 
strategies and processes for self-regulation, all have a commonality of basic assumptions that self-regulation is a 
proactive learning process systemically using metacognitive, cognitive, and motivational strategies to achieve 
academic goals and performance with the interaction of environmental conditions. With self-regulation mechanism, 
expert learners are able to identify the type of tasks and goals, the amount of efforts and time to achieve them, and 
the type of resources and contents to use for accomplishing learning goals. 

Based upon the theoretical grounds, this study defines the critical features of successful self-regulated learning 
such as foreseeing, managing, monitoring and evaluating. When an individual interacts something from external 
environments, he or she foresees what will happen. In academic learning settings, learners’ foreseeing on learning 
tasks and contexts can initiate cognitive and motivational interests. Managing is a process to control external 
learning environments (e.g. time, social interaction and help seeking) and internal cognitive processing. This feature 
concerns with setting one’s own learning goal, making a plan to achieve the learning goal, and managing learning 
resources and tools. Monitoring is the process where learners check out, aware of and think about their own learning 
processes according to the learning goals. Through the monitoring process, learners take responsibilities for the 
achievement of learning goals, and the construction of personal meaning. Evaluating is a process to assess learning 
processes and outcomes for completing entire work. It involves with comparing learning performance based on the 
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predetermined learning goals, and to modify learning strategies, if needed. In actual learning circumstances these 
features are inseparable from and intimately connected to each other. When some approaches to enable these 
features of self-regulation are embedded in learning management system, less self-regulated learners can learn how 
to self-regulate learning process and manage vast amount of learning contents based on the adaptive self-regulation 
features. Even more self-regulated learners can actively manage their learning progress and learning resources more 
effectively based on adaptive learning management system features. 

Ley and Young (2001) also suggest four main principles to embody both effective and flexible guidance for 
self-regulation into instruction as follows: a) guide learners to prepare and structure an effective learning 
environment; b) organize instruction and activities to facilitate cognitive and metacognitive processes; c) use 
instructional goals and feedback to present student monitoring opportunities; and d) provide learners with 
continuous evaluation information and occasions for self-evaluation. Niemi, Nevgi and Virtanen (2003) have 
developed the interactive web-based tool to support learners’ self -regulation in web-based higher education settings. 
The tool consists of three elements: the interactive test bank, tutoring sets, and learning diary. 

In this study, the four main principles are considered for embed self-regulation support strategies in 
learning management systems as follows: 

Individual Preference and Self-control Diagnosis Individual preference and self-control diagnosis builds varied and 
multiple representation of domain knowledge. Multiple representation of knowledge can help learners to interpret 
the knowledge and incorporate it into existing models . Learners interpret the given tasks and environments with 
their different experiences and their existing mental models. According to this view, varied knowledge should be 
provided so that learners can access it with their preference and control. 

Cognitive and Volitional Progress Tracking Cognitive and volitional progress tracking track learnef s cognitive 
and volitional progress for instructional supports. Individual learners’ experience, such as readiness, interests, 
concerns, feelings, and knowledge expressed from individual learners throughout the whole learning process, is a 
critical basis for learners to aware their progress and for teachers to give them with adaptive instructional supports. 

Directive Self Regulation Guide Directive self-regulation guide empowers learners to direct their learning 
throughout individual and social learning process. Learning environments should provide opportunities and 
conditions to enable learners to self-regulate their learning processes by: a) setting their personal goals; b) planning 
learning activities; c) structuring learning situations; d) selecting resources and learning strategies; e) evaluating 
learning processes and products by themselves ; f) revising the processes and g) transferring obtained knowledge to 
the new situations. Through these regulating processes, learners can have successful academic achievement and 
greater responsibility for their learning. 

Self-awareness and Self-assessment Self-awareness and self-assessment encourage self-awareness and self- 
assessment for the learning process. Learning environments should provide learners with opportunities to review and 
appraise learning activities and learning outcomes by themselves throughout the whole learning process. 

Adaptive Web-based Learning Management System Technology for Effective Self-regulation 

In order to actualize self-regulation strategy in the web-based learning environment, adaptive components 
of computer technologies should be implemented. The systems should have both the domain model and the student 
model. The domain model will have the general data bank regarding resources and materials such as topics, 
knowledge elements, objects, and learning outcomes. Meanwhile an individual student’s knowledge model stores 
some value, which is an estimation of the student knowledge level of this concept. This type of model is powerful 
and flexible: it can independently measure the student’s knowledge of different topics. The overlay student model 
can be updated frequently. 

All student actions (the frequency and pattern of visits, the level of problem-solving, the quantity and 
quality of participation) are tracked and used to increase or decrease knowledge levels for involved concepts. 
Another important component of the student model is the model of student’s learning goals. A sequence of assigned 
learned goals forms an individual order of learning. Adaptive guidance technique will provide adequate information 
for step-by-step sequence and progress based on the comparison between domain model and individual model 
information. 

Brusilovsky and Vassileva (2003) suggest the adaptive web-based learning mechanisms for large scale 
web-based education which can potentially be implemented for adaptive learning management systems in terms of 
the glossary, indexed textbooks, advanced navigation, direct guidance and adaptive navigation support, and adaptive 
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guidance for pre-requisite based help. These adaptive mechanisms can be alternatively implemented for self- 
regulation strategies of guiding, monitoring, organizing, and evaluating individual learners study and learning as an 
important part of learning management systems. 

The glossary According to Brusilovsky and Vassileva (2003), glossary is considered as a visualized network for 
various domain of knowledge. Each node of domain knowledge is connected to each other within the glossary. The 
links between domain model concepts consist navigation paths between glossary entries. This glossary function can 
effectively used for monitoring particular learner’s progress of study as well as particular domain of knowledge and 
skills that she/he has been studying. This can provide guidance for further study or pre-requisite study of particular 
learners. 

Indexed textbooks Indexed textbooks are likely to help tracking the current learning progress and schedules. One of 
the biggest problems in web-based learning for large scale of contents and learners is that individual learners can 
have hard time to follow up the consistent study progress due to the lack of monitoring function. However, if the 
indexed textbook function monitors the detailed individual study progress, learners do not need to be intimidated by 
a lot of learning contents. They will be able to catch up with the previous learning progress consis tently. 

Advanced navigation Advanced navigation has the functions of holding the knowledge about the domain and about 
the textbook content to serve a well-structured hyperspace. According to Brusilovsky and Vassileva (2003), this 
system provides all regular navigation tools: sequential and hierarchical links within unit hierarchy. At the same 
time, it provides the navigation center for one-click transfer to all sections on the same or upper levels as well as 
generates a table of contents where all entries are clickable links to the particular content. This function will be able 
to support learners to organize complex learning hierarchy as well as the breadth of learning contents without 
redundant steps. 

Adaptive annotation and direct guidance According to Brusilovsky and Vassileva (2003), to support the student 
navigating through the course, the adaptive system have adaptive annotation and direct guidance technologies. 
Adaptive annotation means that they system uses visual cues (icons, fonts, colors) to show the type and the 
educational state of each link. Direct guidance means that the system can suggest to the student the next part of the 
material to be learned. This function is the highlight of adaptive system to support guiding of self-regulation strategy 
in the adaptive learning management systems. 

Adaptive guidance for pre-requisite-based help This is problem-driven approach rather than goal-driven. Therefore, 
the evaluating steps are embedded behind the function. The system’s knowledge about the course material 
comprises knowledge about what the pre-requisite concepts are for any unit of the textbook. At the same time, the 
current level of knowledge and skills of learners should be evaluated before giving adaptive guidance. When 
students have problems with understanding some explanation or example or solving a problem. In that case they can 
prerequisite-based help (using a special button), and the system generates a list of link to all sections that present 
some information about background concepts of the current section. Adaptive guidance provides significant 
assistance for novices while adaptive navigation support provides significant assistance for more experienced 
learners (Brusilovsky and Vassileva, 2003). 

Personal construction on learning process This system provides with an individual space where learners can 
cumulate and reconstruct inputs and outputs obtained throughout the learning processes by themselves. That is, this 
function is to save every information and data that learners build up, to organize those by the pre -set criteria, and to 
allow learners retrieving and reconstructing at anytime. Various tools, such as communication tools, process tools 
should be included to support learners’ activities. Adaptive Web-based LMS features based on adaptive technology 
discussed above can highlight some of their features based on self-regulation principles and strategy as follows in 
table 1 



Self-Regulation 

Principles 

Individual 
Preference and 
Self-control 



Self-Regulation Strategy 

• Interpretation of individuals ’ 
existing knowledge and 
experience 



Adaptive 

Technology 

• The Glossary 

• Indexed 
textbooks 



Adaptive Web-based LMS 
Features 

• Pre -diagnosis of the current 
users ’ knowledge and experience 

• Analysis of the user 
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Diagnosis 



• Learner identification of their 


• Advanced 


characteristics in terms of subject 


preferences and level of self- 


navigation 


matter interest and needs 


control 




• Self-diagnosis for users ’ 


• Diagnosis of individual 




volitional control and self- 


differences in terms of readiness, 




regulation status 


interests, concerns, feelings, and 




• Automated user analysis report 


knowledge 




• System suggestion and advice for 


• Multiple representation of 




user’s goal setting and planning 


domain knowledge 




• Visualized network presentation 
for various domain knowledge 
relevant to users current needs 



Cognitive and 
Volitional Progress 
Tracking 



Directive Self- 
Regulation Guide 



Self-awareness and 
Self-assessment 



Advanced 
navigation 
Adaptive 
annotation and 
direct guidance 

• Adaptive instructional support 
based on the individual progress 



• Individual tracking for learner’s • 

cognitive and volitional progress 

for instructional supports. • 

• Progress checking throughout the 
whole learning process 



Adaptive 
guidance for 
prerequisite based 
help 

Personal 
construction on 
learning process 

• Resources and learning strategies 
selection 

• Self-evaluation of learning 
processes and products 

• Revision of regulation processes 

• Transfer of obtained knowledge 
to the new situations. 

• Self -responsibility for academic 
achievement 



Direction for individual self- 
regulation strategy 
Practice provision for self- 
regulation 

Personal goal setting 
Learning objective planning 
Learning situation structuring 



• Provision of Self-awareness and 
self-assessment for the learning 
process. 

• Support of learning environments 
for self review and self appraiser 



• Personal 

construction on 
learning process 



• Individual performance gap 
report between current learning 
status and the pre -planned status 

• Individual learning progress and 
current status report for each 
subject matters or course 

• System suggestion and advice 
based on users’ current learning 
management and performance 
status 

• Provision of expert model for 
users ’ self-regulation 

• Support of granular model for 
users ’ practice of self -regulation 
technique 

• Personal goal setting support 
system 

• Learning objective planning 
support systems for individual 
subject matter 

• Resources and strategy selection 
support systems for task specific 
performance improvement 

• Self-evaluation support systems 
based on each task or topic 

• Systems generated feedback to 
adjust the initial setting of goals 
and objectives 

• Post diagnosis systems for 
self-progress assessment 

• Systems generated advice 
report based user self- 
diagnosis 



Conclusion 

The increase of the needs of Web-based instruction is indispensable in this information society due to its 
social, economic, and technological forces. Thousands of web-based courses and other educational application and 
resources are available in various formats but they are not guided, organized, monitored, nor evaluated to meet the 
individual needs and satisfaction. The advanced artificial intelligence technology for adaptive learning (content) 
management systems should be considered so that LMS/LCMS can contribute to raise the individual performance 
improvement and organizational impact to the next level and increase the return on investment. 
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ECIT Program 



Institutions applying for or seeking to continue in l; Alts accreditation and AECT National Recognition will 
use new web-based forms to submit reports on individual programs of study within colleges of education and 
educational technology and school media programs beginning with a pilot study fall 2004. This new way of 
reporting also coincides with a change in evidence required to determine the quality of the program. The change 
streamlines and adds increased consistency to this part of the NCATE accreditation process by using aggregated 
performance data gathered on all candidates in a program (NCATE, 2004). This information will assist AECT 
Reviewers, and those institutions planning to apply for NCATE and AECT National Recognition, in preparing to 
use the new web based format. 

For each program that prepares educators to work in P-12 settings, institutions must identify assessments 
that address the standards for that program. They will also submit the scoring guides or criteria used to measure 
candidate performance on the assessment, and a table with the aggregated results of the assessments. The 
assessments should provide comprehensive measures of candidate content knowledge, knowledge about teaching 
and learning, and candidate effects on student learning. Assessments included on the program report must be taken 
by all candidates in the program. These common measures across programs, a new feature of the review process, 
will increase consistency of the data NCATE receives on programs. 

On the web-based report, the ECIT standards are organized into three different areas of NCATE Unit 
Standard 1: (1) content knowledge, (2) pedagogical and professional knowledge, skills, and dispositions, and (3) 
effects on student learning. Information on the 6-8 key assessments will provide the data needed to meet the 
standards. The assessments must be those that ajl candidates in the program are required to complete and should be 
used by ECIT programs to determine candidate proficiencies as expected in the program standards. Likewise, the 
assessments are grouped into these three areas. This way, information from the program reports can be used in the 
NCATE Unit Standard 1 report, which states that candidates know the subject matter they plan to teach and how to 
teach it effectively so that students learn. Institutions will submit data for the most recent three years and are 
expected to provide the percentage of candidates achieving at each level identified in the scoring guides. 

In fall 2004, for institutions undergoing the NCATE program review process, NCATE and AECT will 
implement a new system for program reviews beginning on a pilot basis . The new process includes the submission 
of 6-8 assessments that provide evidence of candidate mastery of the AECT ECIT standards. Pilot program report 
forms have been developed, and are located in the program standards on http://ncate.org/standard/programstds.htm 
By November, 2004, the forms will be transferred to a web-based program report system. Paper reports will no 
longer be accepted or required. 

The new program report format will be the same across SPAs, although AECT will customize the requirements 

for the 6-8 assessments to conform to the standards and assessments unique to the ECIT standards. 

Assessments must address the five following areas: 

1. State licensure examinations of content knowledge (if applicable) 

2. At least one additional assessment of content knowledge 

3. An assessment of candidate ability to plan instruction, or (for non-teaching fields) to fulfill identified 
professional responsibilities 

4. The evaluation of clinical practice; and 

5. An assessment that demonstrates candidate effect on student learning, or (for non-teaching fields) the 
ability to create supportive learning environments. 
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AECT will provide a training session for reviewers and program submitters at the 2004 Chicago 
conference. Further requests for assistance should be directed to the Chair of AECT’s Standards and Accreditation 
Committee. 

New System Phase In 

Following NC ATE’s new visit formula of once every seven years, all institutions are expected to submit 
program reports one semester before the scheduled NCATE visit, instead of 2-3 semesters before the visit, as was 
previously required. The chart below outlines the submission dates for institutions planning to submit program 
reports through spring 2006. 



Scheduled NCATE visit 


Program reports due 
to NCATE 


Format to use 


Fall 2005 (1) 


2/1/05 


“Pilot” program report form 


Spring 2006 


9/15/05 


Revised report form (2) 


Fall 2006 


2/1/06 


Revised report form 


Spring 2007 


9/15/06 


Revised report form 



Reference 

National Council for Accreditation of Teacher Education (2004). Program standards. Retrieved September 1, 2004 
from http://ncate.org/standard/programstds.htm 
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Introduction 

In recent years, Web technologies have helped expand distance education in higher education. Online 
programs have become widely and generally accepted in many countries (Moore & Anderson, 2003; Ratcliff et al., 
2001). According to the U.S. Department of Education (1999), the number of Institutions of Higher Education 
(IHEs) offering distance education has increased by one-third since 1994-1995; and 44 percent of IHEs offer 
distance courses. The number of enrollments reached 1.3 million credit hours, and thirty percent of all distance 
education courses now use the Web (cited in Swail & Kamptis, 2001). A recently conducted survey found that the 
number of online learners in fall 2002 topped 1.6 million, with public institutions leading the way (having 
approximately 1.3 million online learners) (Allen & Seaman, 2003). Moe and Blodget (2000) further noted that due 
to the increasing demand and the limited access to high-quality postsecondary education in many parts of the world, 
there could be as many as 40 million online education learners by 2025. Given this great increase in online 
education programs, performing evaluation studies on such programs has become imperative (Thompson & Irele, 
2003). The main purpose of this short paper is to describe the evaluation of current online education at three levels: 
(a) macro-level, referring to the evaluation of entire online programs; (b) meso-level, referring to the evaluation of 
individual online courses; and (c) micro-level evaluation, referring to the evaluation of online students’ learning. At 
each level, relevant literature, pertinent evaluation issues, and questions are discussed. While many issues addressed 
in the paper are applicable to both educational and corporate settings, the paper is focused on the higher education 
setting. 



Terminology 

Online Education 

Many terms have been found, both in research and practice, to refer to teaching and learning over the 
Internet. These include distance education and online education. Moore (2003) defined distance education as “all 
forms of education in which all or most of the teaching is conducted in a different space than the learning, with the 
effect that all or most of the communication between teachers and learners is through a communication technology” 
(p.xiv). Distance education, under this definition, then includes non-Internet technologies such as video or 
correspondence learning. Since the three evaluation levels being discussed (i.e., programs, courses, and student 
learning) are all Internet-based, the term online education will be used. 

Evaluation 

There have been many arguments about the correct use of the term evaluation versus the term assessment. 
One position is that evaluation is different from assessment. Some researchers, especially in the North America, use 
the term evaluation to refer to studies implemented to examine and report on the strengths and weaknesses of 
programs, policies, organizations, and the like to improve their effectiveness (American Evaluation Association, 
1999); Assessment, on the other hand, is used to refer to “the formation of value judgments to determine the 
significance, the importance, the value of learning and knowing”, and using “a variety of procedures to obtain 
information about individual’s learning” (Delandshere, 2003). Another position regards assessment as a subset of 
evaluation and a valuable tool in the larger evaluation activity (Rowntree, 1992; Thompson & Irele, 2003). As 
Thompson & Irele (2003) argued, “Assessment asks ‘How much?’ whereas evaluation asks ‘Is it good enough?’ and 
‘If not, why not?”’ In this paper, we used evaluation as a generic term to include evaluation done at the program 
level, the course level, and the student learning level. 
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Macro -level evaluation 

Macro -level evaluation refers to the evaluation of the entire online education program. Macro -level evaluation 
helps ensure the accountability and quality of a program (Fitzpatrick, Sanders, & Worthen, 2004; Rutnam & 
Mowbray, 1983). Such evaluation is important for an online education program for several over-arching purposes 
which include: (a) justifying the investment of resources, (b) measuring progress toward program objectives, (c) 
measuring issues of quality and effectiveness, (d) providing a basis for improvement, and (e) informing institutional 
strategic planning and decision making (Thompson & Irele, 2003). From these general purposes, several pertinent 
evaluation questions can be drawn. 

• Is the program consistent with the institutional mission (Law, Hawkes, & Murphy, 2002)? 

• Does the program have clear goals and objectives, and are there agreed upon provisions and measures for 
program oversight, accountability, evaluation and assessment (Law, Hawkes, & Murphy, 2002)? Agreed 
upon methodologies for evaluation may be models such as I-P-O, CIPP, and AEIOU (Koessl, 2003). 

• Are resources allocated for infrastructure and communication technologies, faculty training and support, 
and student recruitment, retention and support (Sherry, 2003)? Are criteria in place when deciding upon 
the allocation for competing interests? 

• Is the program cost effective (Rovai, 2003; Lorenzo & Moore, 2002; Twigg, 2003)? Can a positive return 
on investment be shown for fiscal costs? 

• Is the technology delivery system as reliable and secure as possible, include plans for backup and disaster 
recovery (IHEP, 2000). 

Meso-level evaluation 

Meso-level of evaluation refers to the evaluation of individual online education courses. Central to this 
level of evaluation is the question of what criteria should be used to evaluate the online courses and instruction. 

Whereas the Seven Principles for Good Practice, developed by Chickering and Gamson (1987), were 
originally intended for undergraduate education, they have been used widely in the evaluation of online courses, 
including those taught at the graduate level (Moore & Anderson, 2003). In addition to the seven principles, Graham, 
Cagiltay, Craner, Lim, & Duffy (2000) added four principles related to human computer interface (HCI) design 
when evaluating four graduate level online courses. Based on a relatively comprehensive literature review of online 
course evaluation, Achtemeier, Morris, and Finnegan (2003) generated eleven evaluation questions and used them to 
examine course evaluation instruments that were employed with online courses or web-enhanced courses from 
thirteen institutions. In addition, other researchers and institutions have developed many guidelines and principles 
for the design and development of courses. Although the guidelines and principles were not developed (only) for the 
purposes of course evaluation, they shed great light on the selection of evaluation criteria (Achtemeire et al, 2003). 
Especially helpful resources are five principles for effective online instruction by Hacker & Neiderhauser (2000); 
thirteen faculty guidelines by Sherry (2003); and an emerging set of guiding principles and practices for the design 
and development of distance education by the Innovation in Distance Education Project (n.d.). 

Based on the available literature and discussion with colleagues, the second author of this paper has identified 
and developed ten key questions (criteria) for the evaluation of online courses, The questions have been employed to 
evaluate an online course offered to undergraduate students (preservice teachers) (Liu, 2003). The ten key questions 
are; 

• Are course objectives, instructor’s expectations, and evaluation criteria of assignments well communicated 
to the learner? 

• Does the course provide students sufficient support (including instructional and technical) for meeting the 
course objectives and other relevant needs of students? 

• Does the class encourage students to be active learners? 

• Does the online course encourage learner-instructor contact? 

• Does the online course encourage interaction and collaboration between and among learners? 

• Does the course encourage meta-cognitive skills? 

• Is learning grounded in effective (contextual, authentic) examples? 

• Is the instructor’s feedback appropriate for supporting learner learning (e.g., not too much nor too little, just 
in time, etc.)? 

• Does the class use multiple evaluation methods aligned with course objectives and designed activities? 
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• Does the course use technology effectively in supporting learning and teaching? 

Further research on meso-level evaluation is still desired (Achtemeire et al, 2003; Bonk, 2003). 

Micro -level evaluation 

The focus of micro -level evaluation is on the individual online learner. A review of the relevant literature 
suggests that evaluation of the individual learner typically falls into one or more of the following three categories: 

(a) the learner’s perception of online learning, (b) the learner’s process of online learning, and (c) the learner’s 
product from online learning. 

Learner’s Perception 

When learners enroll in an online course, they enter into an educational experience very different from the usual 
classroom-based face-to-face environment. Current online education courses are usually mostly or entirely text - 
based, asynchronous, and have multiple discussion threads. As such, course administrators and instructors are 
usually interested to know how their learners “feel” about the course experience. Because of the difficulty of 
reaching online learners who are often separated by space or time, evaluation of learners’ perception typically 
involves questionnaire surveys. Some of the common evaluation questions pertinent to this purpose include: 

• Does the learner enjoy the whole course? 

• What attitudes do the learners have towards online learning before, during, and after the whole course? 

Learner’s Process 

While the learner’s perception of online learning can be helpful and useful information, it is usually not 
sufficient to end just there. Most instructors would also want to know about their learners’ engagement in online 
learning through many different processes such as collaboration, cognition, problem solving and others. One 
common method of determining such processes is through a content analysis of the learners’ online discussion 
transcripts. Table 1 lists some of the content analysis models for evaluating these processes. 

Table 1. Content Analysis Models for Evaluating Student Processes in Online Education 



Processes 


Content analysis model 


Cognitive processes 


Henri (1992) 

Newman, Johnson, Webb, & Cochrane (1997) 


Meta-cognitive processes 


Henri (1992) 


Social construction of knowledge 


Gunawardena, Lowe, & Anderson (1997) 


Collaboration processes 


Murphy (2004) 


Problem solving processes 


Murphy (2004) 



The use of content analysis, however, is not without some problematic issues. Some of these issues include: 

• What kind of unit of analysis should be used - entire message postings, thematic units, or sentences? 

• How is high reliability insured in content analysis? 

• How are “passive” learners evaluated? Passive learners, as found in a study by Sutton (2000), do not 
participate often in the discussion but consider themselves to have learned a lot from reading and reflecting 
on the comments and responses posted by others. 

• How is the validity of the content analysis insured? Do the indicators of the content analysis models 
describe what they purport to describe (Krippendorf, 1980)? 

Learner’s Product 

Typically the evaluation of the learner’s product of online learning is used to determine how much 
knowledge or skills he or she has acquired at the conclusion of the course. This is typically administered in 
traditional ways such as end-of-course tests, final papers, and final projects. Alternative methods include portfolios 
and performances. 
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Summary 

This paper describes three levels of evaluation (macro, meso, and micro) pertaining to online education in 
the higher education setting. Central to the macro -level evaluation is the question of whether the entire online 
education program is able to address five over-arching purposes which include (a) justifying the investment of 
resources, (b) measuring progress toward program objectives, (c) measuring issues of quality and effectiveness, (d) 
providing a basis for improvement, and (e) informing institutional strategic planning and decision making. Meso- 
level evaluation is related to the evaluation of individual online education courses. In this paper, we identified ten 
pertinent criteria that may be useful to practitioners who wish to evaluate at this level. Finally, micro -level 
evaluation focuses on the individual online learner. This level can be divided into three different areas: (a) the 
learner’s perception of online learning, (b) the learner’s process of online learning, and (c) the learner’s product 
from online learning. 



References 

Achtemeier, S. D., Morris, L. V., and Finnegan, C. L. (2003). Considerations for developing evaluations of online 
courses. Journal of Asynchronous Learning Networks, 7(1). Available: http://www.sloan- 
.org/publications/ialn/v7nl/pdf/v7nl achtemeier.pdf 

American Evaluation Association. (1995). Guiding principles for evaluators. New Directions for Evaluation, 66, 19- 
26. 

Bonk, C. J. (2003 Fall). Lectures given in the course P600/R685 Online Learning Pedagogy and Evaluation. 

School of Education, Indiana University: Bloomington. 

Caffarella, R. S. (2002). Planning programs for adult learners: A practical guide for educators, trainers, and staff 
developers. San Francisco, CA: Jossey-Bass. 

Delandshere, G. (2003 Fall). Class handout distributed in the course Y527 Educational Assessment and 
Psychological Measurement. Indiana University: Bloomington. 

Institute for Fligher Education Policy. (2000) Quality on the line: Benchmarks for success in Internet-based 
distance education. Washington, DC: The Institute for Higher Education Policy. 

Graham, C., Cagiltay, K., Craner, J., Lim, B., and Duffy, T. M. (2000), Teaching in a Web based distance learning 
environment: An evaluation based on four courses. CRLT Technical Report No. 13-00. Available: 
http://www.crlt.indiana.edu/publications/workpapers/crltOO- 13.pdf 

Hacker, D. J.,& Neiderhauser, D.S. (2000). Promoting deep and durable learning. In R.E., Weiss, D., Knowlton, & 
B., Speck, (Eds.), Principles Of effective teaching in the online classroom [Special issue]. New Directions 
for Teaching and Learning, 84. 

The Innovations in Distance Education Project (n.d.). An emerging set of guiding principles and practices for the 
design and development of distance education. Retrieved December 10, 2003 
from http://www. worldcampus.psu.edu/ide/docs/guiding principles.pdf 

Koessl, D. (2003) Review of research and research resources related to distance education and program 
evaluation . Retrieved October 8, 2004 from 

http://cstld.colostate.edu/filesanddocument/CLTDBibliographvl.pdf . 

Krippendorf, K. (1980). Content analysis: An introduction to its methodology. Beverly Hills, CA: Sage. 

Law, J., Hawkes L., & Murphy, C. (2002). Assessing the on-line degree program. New Directions for 
Teaching and Learning, 91, 83-89. 

Liu, S. (2003). Evaluation of an online course. An unpublished manuscript. 

Lorenzo, G. & Moore, J. (2002). The Sloan Consortium report to the nation: Five pillars of quality online 
education. Needham, MA: Sloan Center. 

Moe, M. T., & Blodget, H. (2000). The knowledge web: Part 1. People power: Fuel for the new economy. New 
York: Merrill Lynch. 

Moore, M. (2003). This book in brief: Overview. In M. Moore & W. Anderson (Eds.). Handbook of Distance 
Education (pp.xiii-xxiii). Mahwah, NJ: Lawrence Erlbaum Associates. 



246 



Integrating Technology in Classrooms: We have met the enemy and he is us. 

Brad Hokanson 
Simon Hooper 
University of Minnesota 

Introduction: Technology, computers, and schools now; failings and doubts 

At the cusp of the millennium and in a period marked by the availability of computers and by an 
information explosion, technology spending in education is at an all time high. At present, U.S. K-12 technology 
budgets exceed $3.5 billion annually, and further increased spending has been proposed. A recent government report 
recommended increasing that budget to approximately $13 billion, the equivalent of five percent of all K-12 
spending (President's Committee of Advisors on Educational Technology, 1998). Currently, however, computers 
remain apart from the mainstream of the curriculum and are not integrated, as say, simpler technologies such as 
books or overhead projectors. 

The impetus for computer use in schools is driven in large part by changes in business practices (Cuban, 
1995, Kerr, 1996). Indeed, much of the recent economic boom was considered to be driven by computer-encouraged 
productivity increases (Drucker, 1999). While technology has transformed the manner in which businesses operate, 
its impact in schools has been modest. While there is a belief that computers could improve educational productivity 
and help schools to teach more efficiently, evidence to support this belief is scarce. Indeed, the educational 
landscape is littered (figuratively, if not literally) with the debris of failed technologies. 

An emerging backlash has begun to criticize computer use in schools. Some suggest that technology 
spending has been responsible for diverting limited financial resources from more worthy causes and the emphasis 
on the use of computers has been blamed for constricting the quality of educational experiences. There have also 
been complaints about the vocational focus on the development of comp uter-based skills needed in the business 
community rather than on the academic needs of individual students (c.f. Oppenheimer, 1997). 

Heeding these criticisms, it is important to consider the potential of computers in education. Given the 
limited impacts of earlier infusions of educational technology, there appears to be little cause to believe that 
contemporary technology will be more effective than its predecessors. There remains, however, a belief among 
some educational technologists that computers have the potential to transform educational practice. 

The notion that students can learn better with computers is based on the belief that a relationship exists 
between technology and knowledge. However, this relationship is frequently misunderstood. Technology, by one 
definition, is an embodiment of knowledge (Saettler, 1990) and, significantly, we also use a wide range of 
technologies in our pursuit of knowledge (Clark, 1997). Recognizing and understanding the relationship between 
technology and learning, from the pencil to the computer, should help us improve our educational systems. 

The value of the computer in education may be clarified by distinguishing between using it as a 
transmission device or as a learning device. We believe that too much emphasis has been placed on learning from 
technology (e.g. viewing educational television, completing computer drills and tutorials, etc.) rather than learning 
with technology. 

The difference, beyond semantics, is critical to technology's integration into the curriculum. When 
technology is used simply to deliver content, the only educational benefits that can be anticipated are improved 
access and educational efficiency. In contrast, improved educational effectiveness requires using technology to 
enable learners to explore, expand, and to enhance their own capabilities; i.e. to create their own knowledge. Instead 
of using technology to deliver educational materials, our goal should be to develop learning environments in which 
students more effectively generate knowledge using the technology (Jonassen, 1996; Hannafin, 2000). 

It is this generative capability of computers that powers our argument that computers should be integrated 
into the curriculum. Our end goal in education, and as educational technologists, should be to develop and instill 
knowledge, not information. 

In this paper we will consider the potential of technology in education, examine what it means to 
"integrate" technology in the classroom, outline barriers to technology integration, and consider several implications 
for effective technology use. We begin by noting doubts regarding computer use in schools and outlining the 
potential of computers. Next, we define the term "integration" and examine various levels of educational technology 
use. Subsequently, we consider the idea of integration through the use of the metaphor of human rights; learning 
from the barriers and problems of the Civil Rights Movement. We examine barriers to computer integration into the 
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curriculum and conclude with several imp lications of our observations that are intended to guide teacher education 
and support professional development for in-service teachers. 

The nature of integration 

Computer integration occurs along a spectrum of effectiveness and involvement. At one end, computers 
may be available to assist with various tasks, but their impact on educational activities is minimal. At the other end, 
a lively curriculum may fully integrate their use as part of normal growth and change. In practice, most teachers fall 
somewhere in between. 

Hooper and Rieber (1994) noted five points in the passage to integration: familiarization, utilization, 
integration, reorientation, and evolution. Beyond an examination of technology, these layers also illustrate a view of 
education in general. They suggest the extent to which a teacher -- or curriculum -- is willing to adapt and innovate. 

Familiarization occurs when teachers are initially exposed to a new technology. At this stage, technology 
use is not extensive, and little or no impact is made on the curriculum. 

Utilization involves using technology to support and improve the efficiency of the existing curriculum. At 
this level, computers may simply be used to perform tasks faster and more efficiently. However, qualitative changes 
in the curriculum are few. As the instructional method does not change, little change in performance should be 
expected. For example, drill and practice lessons on a CD-ROM are probably about as effective as the same lessons 
printed in a book. Over the history of education, diverse technologies have been used primarily for functional 
efficiency, that is, having more students exposed to the same lecture and having lessons codified for broad 
distribution (Saettler, 1990). 

The integration level is marked by the inclusion of activities that could not be attempted without the 
technology. For example, the computer may act as an enabling force in the curriculum, allowing new instructional 
methods to be used. Using the computer as a cognitive tool (Jonassen, 1996) in the assembly, examination, or 
presentation of materials by learners illustrates this type of use. 

Reorientation represents a scalar difference in technology use; substantial and significant computer use 
occurs, causing a re -directioning of the curriculum. Not only may different instructional methods be attempted in the 
classroom, but the entire curriculum may adopt methods only possible through computer use. One such example is 
the use of a specific software package in all the classes across a school, and an attendant redesign of the area 
curriculum. Using Geometer's Sketchpad for all geometry classes, with assignments, exams, and scope all 
reexamined for use with the software represents such a reorientation. 

At the evolutionary level, the specific technologies become less important. Rather, media are differentiated 
by their capabilities for different methods of instruction. What is significant is the level of innovation, adoption, 
change, and improvement that occurs in the classroom. 

To illustrate how a teacher at the reorientation or evolutionary levels might use technology, imagine how 
this person might integrate the potato as a technology. When would the potato be considered truly integrated into the 
curriculum? When some teachers use potatoes? Or when the curriculum is reorganized to include the potato, and all 
its capabilities, while teachers continue to develop and evaluate its role in the curriculum. One could imagine 
counting with potatoes, making art with potato stamps, using potatoes to demonstrate galvanic or osmotic concepts 
in chemistry, and the potato as the centerpiece in studies of American and Irish history. It is not the specific 
vegetable that is important for effective use, but rather how the potato is used, and how the teacher and curriculum 
can take advantage of the unique characteristics of the potato. Moreover, having enough potatoes is not the problem; 
it is more important to focus on how instructional methods can take advantage of the specific capabilities of the 
potato to stimu late learning. 

The concept of vegetables being fungible, i.e., interchangeable, is an extension of a well-known metaphor 
in the educational technology field, Clark’s (1983, 1994) grocery truck. He argued that as the truck has little 
influence on the nutrition derived from groceries, the media delivering instruction makes no difference in learning. 
However, the manner in which a technology is used governs its educational potential. Inherent in the method of 
strictly comparing technologies by common capabilities, Clark ignores their unique capabilities. A teacher at the 
evolutionary level could develop exceptional student learners with a potato, while a less developed teacher using 
high end Silicon Graphics workstations would achieve about the same results as with a simple overhead, in just 
about the same manner. Different technologies, whether potatoes or computers, have specific advantages that must 
be recognized and used to fulfill their potential. It is the recognition of the specific advantages that drives 
improvement in instructional method. 
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Exploring meaning: Integration as metaphor 

The term integration may provide us with a useful metaphor to guide technology use. American Society 
has a strong connection between the term 'integration' and the Civil Rights Movement of the late 20th Century. 
Integration is a multifaceted process involving equal rights, equal opportunity, and full involvement into society for 
all citizens, regardless of race; it is a movement that remains in process. We may be able to use cursory observations 
of the integration process and the terms ghetto and tokenism to shed light on how technology can be integrated into 
the curriculum. 

Through painful experience, we recognize different levels of integration; integration compelled by law or 
by edict often falls short of true and complete integration. When integration is forced on society, only token 
participation (i.e. minimal and by quota) often occurs. When computers are restricted to a ghetto, value is limited for 
the curriculum as a whole. 

Tokenism is clearly evident in how some schools have used computers. For example, some school districts 
provide a computer for each classroom. While such a policy ensures that all classrooms have computer access, little 
substantive change can be expected when individual access is so minimal. As with token racial integration, only 
limited change occurs. 

One can also imagine a curriculum where computers are ghettoized, located far from the mainstream 
learning activities. They are not a part of the regular schooling, but held as a separate, albeit special, element in 
schooling often called "computer labs". Here as well, there is no integration, but what is more accurately described 
as segregation. Access is controlled, limited, and divergent and informal uses denied. Clearly, integration into the 
curriculum will also not occur through the computer ghetto. Exceptional results may occur in limited areas, but that 
progress will not extend across the curriculum until the computer leaves the single -use room, and is dispersed 
physically and pedagogically . 

The goal of an integrated society builds on the notion that all races freely interact, and the values, cultures, 
and creations of each is shared and cherished. So too, we envision a curriculum where technology use is determined 
by its capability to support learning. In such an environment computers would be used as an active part of the 
classroom; where technology is not a special event, but rather as a normal part of the classroom and curriculum. 

Obstacles to computers integration 

Given the substantial investment in computers and internet access, what are the impediments that remain to 
full and vibrant use of computers in the curriculum? Presumably, there is more to effective and integrated use of 
computers than simply installing the technology. 

Ertmer (2000) addresses technology integration and identifies two sets of obstacles in education. They are 
first order obstacles . which include problems involving hardware, access, and technical support; and second order 
obstacles , such as changes in pedagogy, or personal preferences that influence an individual's acceptance of new 
ideas. 

First order obstacles are clearly quantifiable and demonstrate the level of our commitment to new 
technologies. They are at the forefront of both our current engagement with technology and the belief that computers 
will improve education. Society is working diligently to overcome these obstacles as demonstrated by the number of 
computers installed, and the quest to wire all schools to the Internet: 86% of all K-12 students have access to 
computers either through home or school; over 95% of all schools have computers installed (NPR/Milliken, 2000). 
With “5 zillion” computers already installed in the K-12 educational system, is it any wonder that vast 
improvements in the quality of education are anticipated? 

Yet, it is an examination of second order obstacles that may explain our failure in this "Great Leap 
Forward" and provide insights into potential success. The real need for change is not with access to the machines, 
support or software; the 'problem' is within. It is pedagogical, curricular, and methodological. The concept of 
technological integration into the classroom may be as much about evaluation, change, and reflection of the 
curriculum as it is about computers. It's personal: We have met the enemy and he is us (Kelly, 1954 et. seq.). 

The computer will affect how the curriculum is organized, what is taught, and how instruction is 
conceptualized, and structured. They will affect how and what people learn. 

Implications 

As discussed, successful integration is more than a story of numbers. It involves a series of changes and re- 
orientations that are less dependent on the number of computers in a school or classroom, and mo re dependent on 
other human factors. While teachers use computers at about the same rate as the rest of society (Cuban, 2000), 
integration of the computer into the curriculum often lags primarily due to soft factors (Ertmer, 2000). While 
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teachers use computers for managerial tasks, it appears that use has not extended to their instructional methods. The 
implications of these observations may be summarized into a series of heuristics: 

It's the method, stupid. (With apologies to James Carville). The critical element in any educational environment is 
the instructional method, not the means by which it is delivered. Computers, whether integrated or not, are not the 
sole determinant for the curriculum nor do they guarantee success. The questions behind the integration of 
technology into the curriculum are not solely about computers; they are also about educational improvement. 

Attributing improvement or lack thereof to the use of computers is a simplistic answer to questions of 
educational quality; the true source of success is woven into the entire fabric of the educational enterprise. The value 
of computers comes from their enabling new instructional methods and procedures. 

Central to the effective use of computers is an understanding that computers affording new and different 
instructional practices in teaching. Traditional and existing methods won't work any better through computer use. 
Merely changing the delivery medium will not improve quality. What must change is the instructional method. 

Imagine the differences between a creative, skilled teacher without computers and a less endowed teacher 
with computers. Which teacher would we rather be with in a classroom? Clearly, the former would be more effective 
on a number of levels. The creative and skilled teacher utilizes the materials at hand to present the curriculum in an 
appropriate, engaging, and intriguing manner. 

Start at the beginning. Although it is necessary to place computers where they will be used most effectively, it is 
critical to develop the skills and attitudes of new and in-service teachers toward their effective use of computers in 
the curriculum. Our primary ability to influence education on a broad scale lies with the preparation of new teachers, 
and through that route must come improved and integrated uses of computers as cognitive tools and media. 
Computers, like teachers, must encourage learners' skills to investigate and involve themselves in their studies. 

Technology courses in teacher preparation programs can help alleviate new teachers’ technology fears, 
provide them with new skills for their work, and possibly, give new teachers the ability to instill those computer 
enhanced investigative skills in their own studies. 

Critical to the enhanced use of computers and their true integration into the curriculum is the development 
of teachers' skills to levels beyond introductory use. Teachers must develop skills beyond the rote use of simplistic 
applications. (For example, use of a computerized grade book is faster, more accurate, and more efficient, but 
merely improves an existing process; it does not change the nature of the activity or the educational process.) 

In our educational system, we learn to use writing, reading, mathematics, and scientific inquiry as media 
for enquiry (Hokanson & Hooper, 2000). Used in a representational manner, writing, for example, can be used to 
record the verbiage of the classroom. Writing used in a generative manner, to expand and explore a classroom 
discussion, is much more effective at encouraging learners to develop new ideas and stimulating retention of the 
material examined. Engaged learning, where the student is motivated and involved in developing ideas as part of the 
learning process, is more effective at developing cognitive skills and information retention. Engaged media, where 
the unique characteristics of the media are used to investigate and generate will be similarly more effective. 

The future of technology is not to make education easier, but rather to make learning more effective. Technology 
use often focuses on making education easier and more efficient. Yet learning requires effort and is often time 
consuming. For example, we retain more information when we are forced to use or manipulate ideas: Strategies that 
force us to summarize, restate, or invent have important cognitive benefits. Similarly, higher levels of cognition also 
benefit from mental exertion (c.f. Jonassen, 1996, Salomon, 1983, Kozma, 1991). 

Imagine the best teacher we have experienced. Did that teacher make learning easier, or or require less 
work and involvement? Or did that teacher, in some way, get us to work harder, to put in more effort and thought, 
and thereby to develop our own skills and understanding? 

As with the best teachers, educational computer uses should require that more cognitive effort goes into the 
computer (i.e. is provided by the learner) than comes out (i.e. is delivered by the system). Learners should provide 
ideas, structures, information, and in some cases motivation to the learning process. 

Conclusion 

We began this investigation with several commonly cited criticisms about contemporary uses of computers 
in education. Although much money has been spent, and much effort expended to install computers and internet 
access in to the nation’s schools, few improvements in educational outcomes have occurred. We believe that 
computer technology has not been used effectively, following the pattern of other earlier technologies used for 
education such as television, film, and radio. 
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This notion of effective technology use parallels recent theoretical developments in our understanding 
about how people learn, in particular the shift from instruction to knowledge construction, i.e. the understanding that 
learners create their own knowledge. Using computers in a generative (as opposed to representative) manner is 
central to effective educational use. That is, the most effective methods require users to actively investigate, create, 
and generate ideas, rather than to watch passively. 

The range of computer technology integration is broad and includes different levels of involvement and 
use, which are tied to a curriculum and instructor's ability to accept and use new technologies. Given this extensive 
and diverse range, it is logical to believe that there are more impediments to computer integration than simple 
hardware access; recent research by Ertmer (2000) supports this implication. We found similar impediments to 
integration in our comparison to civil rights integration; while numbers and dictums may mandate and quantify 
certain actions, the soft and subjective beliefs of a society (or a curriculum) restrain progress. The most substantial 
impediments to computer integration are subjective and informal, restricting evolution of education. 

Our implications focused on two primary areas of required change. First, progress in the area will occur 
indirectly, and will be influenced through the development of pre -service and through improvement of in-service 
teachers. The ability - or acceptance - of a new or different technology into a given curriculum is based on the 
understanding and nature of the teachers. Our second area focused on the nature of computer use and needed 
curricular evolution. Any technology may be used to improve education, but it is the nature of that use that makes 
change occur. 

As educational technologists, we also know that the changes that can be wrought with the varied 
technologies are not the panacea for all the ills of education. The lack of effective use may be a symptom of larger 
changes needed to the system. It's not just about computers, it's about education. Whether teachers accept 
computers. . .or potatoes. . .or any other cognitive enhancement is derived from the culture of their school, from the 
culture of their society, and from their understanding of the nature of teaching and education. 
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An Online Simulation in Pediatric Asthma Management 

Keith B. Hopper 

Southern Polytechnic State University 

Introduction 

The Centers for Disease Control (CDC) estimates that nearly 20 million Americans suffer from asthma, 6.3 
million of which are children(Centers for Disease Control and Prevention, 2004). It is not merely an annoyance 
disease, as is commonly believed. Asthma kills. It takes more than 5,000 American lives each year) Asthma Statistics 
in America, 2004) . Asthma is the most common chronic disease of childhood, and it is the number one cause of 
hospitalization and absence from school. African Americans suffer asthma at three times the national rate, and are 
four times as likely to die from asthma as whites. Asthma in Atlanta’s inner city children has increased to alarming 
levels in both morbidity and mortality, especially in black and Hispanic populations. Incidence and severity of 
asthma is inversely related to socioeconomic status. It is a dangerous pulmonary disease, which not only spoils 
quality of life for asthmatic children and their families, but results in many deaths. An important social hazard of 
childhood asthma in children is increased school absenteeism. The CDC estimates that asthma caused fourteen 
million missed schools days in 2002. Loss in productivity by working parents caring for children missing school 
due to asthma is estimated at $1 billion per year) Asthma Statistics in America, 2004). Absenteeism from school 
directly contributes to increased drop-out rates, with lifelong repercussions in earning capacity, health, and quality 
of life. More important, physicians, nurses, respiratory therapists, asthma case managers, and school nurses know 
very well that beyond the startling statistics lies the personal suffering of many individual children. 

For children with asthma, the key to survivability and day-to-day participation in a satisfying life is 
judicious, timely management of the disease. Children with asthma must learn to monitor their breathing condition, 
avoid and manage environmental asthma triggers, and take prompt, aggressive action when needed. Asthma 
management includes monitoring symptoms and status, home treatment with medications and therapy, identifying 
and minimizing environmental asthma triggers, and knowing when and how to seek medical attention. Teaching 
asthmatic children to monitor and manage their disease is a complex instructional goal that must be accomplished to 
a high standard . Sensible management reduces hospitalizations, improves the quality of lives for both asthmatic 
children and their families, saves enormous amounts of money, and saves lives. Preventing a single hospitalization 
involving need for mechanical ventilation saves many thousands of dollars, and possibly a life. But community 
education for asthmatics is often not uniform, not timely, and not readily available. 

Branching-Logic Clinical Simulation 

Certification of competence, and professional credentialing of practitioners in medical fields, especially 
physicians, has been a challenge for more than a century. Beginning in the 1960s, testing and measurement 
researchers attempted to devise workable clinical simulation techniques to measure skills and competencies beyond 
rote memorization of facts(McGuire, 1995). The intent is to reliably, realistically measure practitioner competencies. 
The University of Illinois developed branching patient management problems (PMPs), paper-based simulations 
describing patient complaints and presenting a choice of management options such as initiating therapy, gathering 
additional information, or discharging the patient. The clinical problems proceed through a customized pathway 
determined by patient management choices, and are termed branching-logic simulations. 

The field of respiratory care, an innovator in technology integration in teaching among health care fields, 
has used branching-logic clinical simulations in the credentialing process for respiratory therapists since 
1979(Hopper, 2004). Originally using a latent image paper-based technique, the National Board for Respiratory 
Care (NBRC) replaced paper technologies with computer-managed credentialing exams beginning in 2000. 

Designed to simulate clinical practice reality, NBRC exams are text -based, and provide brief descriptions of clinical 
scenarios, followed by multiple -choice questions. Simulation sections are of two basic types: 

1. Information-gathering, wherein the user requests specific items of information such as patient 
temperature, heart rate, breath sounds, and lab values, and receives the requested data. Items include valid, neutral, 
and harmful selections, relevant to the medical case at hand. 

2. Decision-making, wherein the user assimilates information from the clinical scenario and one or more 
information-gathering sections, and chooses a course of action. Choices made in decision-making sections determine 
the pathway through the remainder of the simulation, and the simulation is in this way customized for each user. 

This provides for alternative but equivalent treatment choices to accommodate regional differences in medical 
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care(Kernaghan, 1978), and allows opportunities to rectify decision-making errors. In this way, users can partially 
redeem their scores by recognizing the results of incorrect choices and taking remedial action. 

In both section types, items are assigned a score between three and negative three, with a score of zero 
earned for neutral items (neither harmful nor helpful). For example, choosing to perform the Heimlich maneuver in a 
choking emergency would earn a score of three points, while choosing to wait for laboratory results in the same 
situation would earn a score reduction of three points. Requesting patient skin tone, in many situations neither 
helpful nor harmful, and which is done quickly and at no expense, is an example of an item typically scored as zero. 
The simulation compiles cumulative scores in both information-gathering and decision-making, and users must 
achieve a calculated cut score in both categories to pass the simulation. The intent of the clinical simulation exams is 
to test the user’s knowledge base, clinical acumen, and critical thinking(Hill, 2002; Mishoe, 2002, 2003). Successful 
users gather appropriate information, without wasting time in an urgent situation, without causing unnecessary harm, 
and without incurring excessive expense. Based on this information, the user then selects a judicious course of 
action, and the simulation proceeds to another section, which would be a likely result in a real clinical arena. The 
branching-logic clinical methodology is widely held to be both challenging and instructional, despite criticisms that 
simple memorization of clinical protocols may falsely verify true critical thinking capacity(Hopper, 2002). 

Project Vision and Scope 

Asthma management is a tightly defined instructional goal, which lends itself well to traditional methods of 
systematic instructional design(Dick & Carey, 2001), and to delivery via Internet technologies. Information on 
asthma and its management changes rapidly. A potential solution was envisioned as a high quality, engaging online 
simulation, which asthmatic children could access from any school, home, church, or business with a rudimentary 
personal computer (PC) and an Internet connection. The child would take charge of the management of a virtual, 
online asthmatic child, and make virtual decisions about environmental triggers, drug administration, self 
assessment, and other basic issues. The simulation would follow a branching-logic format, similar to the NBRC 
credentialing exam for respiratory therapists, and with multiple pathways, so that the learner would encounter 
situations and information customized for his or her knowledge and skills. The simulation could adapt to a variety of 
user ages and circumstances, and print certificates of achievement. Above all, virtual asthmatic children and their 
homes and environments in the simulation should be representative of the users’ circumstances and experience. It is 
essential that target learners relate to the simulation content and images. 

Although there are abundant informational resources on asthma available on the Internet, and sophisticated 
high bandwidth multimedia products published on compact disk (CD), an online simulation on asthma management 
could not be found. An Internet mediated branching-logic simulation in pediatric asthma management, based on the 
NBRC credentialing examination for respiratory therapists, was adjudged worthy of development and investigation. 
Such a project would sidestep the technical disadvantages of production and distribution on CD. Most notably, the 
rapid evolution of content related to asthma management makes CD production a dubious distribution choice. It is 
not workable to recall thousands of CDs when the subject content in asthma management changes; however, an 
online asthma simulation website may be modified and updated quickly, cheaply, and from a single production and 
access point. Additionally, it was anticipated that the branching-logic simulation approach, delivered online, would 
be challenging and engaging for young learners. 

The simulation was envisioned as targeting those in the community most challenged in managing the 
disease. The simulation must be constrained by the lowest hardware standards of potential users, but offerthe 
advantages of being universally, instantly available, and modified as asthma management evolves. Such a project 
has the potential to affect an audience beyond the greater Atlanta area. It is not the intent of the simulation to 
provide comprehensive and complete information on asthma management. Rather, the project was envisioned to 
provide an important supplementary role to traditional training, as would be delivered in the physician’s office, by 
the school nurse, by information documents provided by various organizations such as the American Lung 
Association, and by asthma case workers. This is in keeping with the findings of some researchers that online 
instruction may be best suited to a supplementary role(Harmon & Jones, 1999; Hopper, 2001). Rather than 
attempting to teach everything about asthma, the simulation covers the most important and most common content 
areas affecting asthma management. 



Learner and Context Analysis 

The primary learner population targeted for this project is inner city asthmatic children in the metro Atlanta 
area (Atlanta Empowerment Zone), but also comprises parents and families of asthmatic children, school nurses, 
coaches, physicians in general practice, medical clinic personnel, nursing and allied health students, and other adults 
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in contact with asthmatic children. Asthma case workers from Atlanta’s Zap Asthma organization 
( http://www.sph.emorv.edu/zapasthma/DefauIt.htrr i) were visited and interviewed to determine target learner 
characteristics, and technology access. These SMEs reported that the majority of asthmatic children in this target 
group had at least limited access to PCs and the Internet. Some of the children have access to home PC equipment. 
Most have limited access to technology in school libraries and classrooms, and from community libraries and 
churches. The Zap Asthma SMEs believed that hardware and software standards were dated, as is their own 
computer technology. The SMEs reported that they perceived the target population of children to be enthusiastic 
about computer technology, but with undetermined PC skill levels. 

Design and Development 

Prior to selection of technologies, instructional analysis, and user interface design, project specifications 
were developed: 

• Simplicity — intuitive, robust, and user friendly. 

• Technology soft — technology structure Spartan. To make the simulation available and useable on the 
widest possible range of hardware, the technology structure must be kept minimal Technologies requiring 
high Internet bandwidth, plug-ins, or Flash are excluded. 

• Content accurate. This is the point of the simulation — accurate, current information. 

• Learner centered — language and operation matched to the target audience. 

• Scalability — to accommodate content and operational updates, and to facilitate development for other 
learner groups. 

• Aesthetics — visually appealing, interesting, and perceived as a positive experience. 

• Flexibility — choices and alternate pathways to accommodate users of various ages and genders, and 
multiple accesses by the same users. 

• Feedback — the simulation must provide liberal informational and attitudinal feedback, and provide 
remedial feedback skillfully and constructively. Feedback should be provided throughout the simulation 
experience, and a final summary of performance should be provided. 

Specifications of simplicity and reliability require a minimalist approach to technology selection. The basic 
online platform consists of HTML and Javascript, usable by virtually all current and recent PC workstations. 

The instructional analysis (see figure 3) was performed by two professional instructional designers from the 
corporate arena (Home Depot and Cingular Wireless), under the direction of a faculty instructional designer (project 
lead) who is a registered respiratory therapist (RRT), as an independent study project toward a master’s degree in 
technical communication. The design team conducted subject matter expert (SME) content sessions to extract and 
refine the skills and specific knowledges of pediatric asthma home management. SME sessions included hospital 
health educators, pulmonary rehabilitation specialists, case managers, pulmonologists, and health professions 
faculty. 

The instructional design team first developed and refined the project’s instructional goal: 

Asthmatic children in the greater metropolitan Atlanta area will improve asthma self-management by 
applying preventative, monitoring, and treatment tools and techniques to significantly reduce emergency room 
visits, school absences, and hospitalizations. 



The simulation storyboard was developed in Microsoft PowerPoint (see Figures 5 and 6), in consultation 
with project SMEs. A Southern Polytechnic faculty member in graphics design was contracted to develop 
customized images for the various simulation sections, rendered as smallest possible GIF files (see Figure 1). 
Southern Polytechnic students and faculty in technical communication voluntarily performed much of the technical 
and development work, including technical writing and editing, technical development, and usability testing. 

The website was constructed in HTML and Javascript (see figure 7), using the following development tools: 



Dreamweaver MX 


Macromedia 


website design and management 


CourseBuilder 


Macromedia 


Javascript interactions 


PowerPoint 


Microsoft 


storyboard 


Javascript 


Microsoft 


custom event tracking and scoring 


Photoshop 


Adobe 


photographic images 


Illustrator 


Adobe 


graphics 


Fireworks 


Macromedia 


selected graphics — including animated GIF movies 
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Premiere 


Adobe 


digitized video 


WMA Workshop 


Litex Media 


sound file editing and conversion 


Project 


Microsoft 


project planning and management 



The project was completed in three phases: 









Phase 2 


2002 


storyboard 

SMEs review and refine 
develop prototype 


Phase 3 


2003-04 


code 

graphics and sounds 

usability testing 

deploy (November 1, 2004) 



Major grant funding was provided by GlaxoSmithKline on completion of phase 2. 

The website is designed for access using a Windows PC with Microsoft Internet Explorer 5.0 or later. It 
does not work properly using Macintosh platform or Netscape Navigator browser. This is compatible with 
technology access of the learner population, revealed in learner analysis. Narration sound files were converted to 
smallest workable file size for rapid download. Similarly, motion video was restricted to two simulation sections 
only, and rendered in animated GIF format. Graphics, sound, and motion video files are preloaded in all simulation 
sections to achieve a smoother operational flow for the user. 

Virtual subjects for the simulation were recruited from learners in the target population (see figure 2) and 
are children from metro Atlanta who have asthma. The narrator is a Zap Asthma case worker whose voice and 
language are representative of the cultural environment of the target population. It is recognized that language 
decisions in the project may make the simulation challenging outside the target population, but the specific needs of 
the project’s learners prevailed. Language throughout the simulation project was carefully edited to achieve a 
Flesch-Kincaid grade level average score of about grade five. 

Unlike the NBRC branching-logic simulation format, this project does not conpile separate information- 
gathering and decision-making scores, but calculates a single combined score. Users completing the simulation in 
testing mode earn one of three performance ratings (see figure 8): 

• Asthma Zapper First Class 

• Asthma Zapper Second Cla s s 

• Asthma Zapper in Training 

Final score is reported and users are invited to try for a higher score. 

Field Testing and Usability Testing 

The project was field tested at Atlanta’s Zap Asthma headquarters, with twelve metro Atlanta asthmatic 
children, as part of a one week asthma camp conducted by Zap Asthma case workers. Children were assigned to 
work individually on computer stations with color monitors, external speakers, and DSL Internet connections. PCs 
at this site are surplus and donated units from various federal and state agencies. Average PC age is about five years 
(processor speed and memory undetermined), and all use Microsoft Internet Explorer 5.0. Users ranged in age from 
seven to fourteen years, and most were female. The field test was conducted by the project lead and three volunteer 
graduate students (Southern Polytechnic Technical Communications Program). Major findings from the field test 
included: 

• Users in most cases demonstrated significant computer skills (mousing, typing, browsing, etc.), and 
navigated the simulation website with apparent ease. 

• Instructed to access the review option of the simulation for as long as they wanted, then attempt the testing 
option, users averaged about ten minutes in review mode followed by about thirty minutes to complete the 
exam. The majority of users earned an “Asthma Zapper First Class’’ rating on first attempt, attesting to the 
rigor of content of the Zap Asthma camp preparation. 
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• Users experienced no major technical malfunctions, revealing the simulation to be robust and reliable. 
Download time, even in simulation sections with motion video files (animated GIF) and many sound files, 
did not appear to be a factor in user satisfaction. 

• Voice narration in review mode was clearly an important factor in user satisfaction. Users tended to avoid 
reading remedial text, but listened carefully to audible narration. This is in keeping with current research on 
the use of audio in multimedia learning(Barron & Calandra, 2004). 

User comments were uniformly positive, and in many cases enthusiastic. Asthma case workers monitoring the 
exercise expressed similar positive responses. 

• Some users as young as seven years successfully used the simulation, although the minimum recommended 
age based on field testing is eight years. 

Usability testing was conducted in the Southern Polytechnic usability lab as a culminating exercise for an 
undergraduate course in usability testing. Three learners representing the target population, two parents of asthmatic 
children, and other volunteers accessed the simulation website under controlled conditions. User actions, verbal 
comments and questions, and observer notes were recorded for each participant. Users completed a preliminary 
survey to gather demographic data, and information on the user’s previous knowledge and experience in asthma 
management, and in PC skills. They were debriefed following the simulation to probe user attitudes regarding the 
instrument and the experience. Usability testing revealed many important positive and constructive aspects of the 
simulation, including: 

• The fundamental premise of the simulation was easily grasped and applied by users. 

• The navigational design and structure was confirmed — users successfully entered, participated in, and 
completed the simulation without undue difficulty. 

• Some screen design elements were altered as a result of usability testing. For example, it was observed that 
the first round of participants rarely pushed a “Reveal correct answers” button to self evaluate performance. 
But this improved dramatically in the second round when the location of the button was moved to be near 
the “Go” button that exits each section. 

• Several language problems were discovered and corrected. For example, few of the users understood the 
word “sputum” and this was subsequently defined within context. Similarly, words such as “assemble” and 
“disassemble” were replaced with “put together” and “take apart.” 

• Need for an on-screen switch to toggle sound one and off was apparent (see figure 7). 

• Users expressed emphatic appreciation of voice narration rather than reading the same remedial text. This 
was the case with both children and adults. 

Deployment 

The simulation is scheduled for public release November 1, 2004. A working version of the project may be 
seen at the following URL: 

http://www.spsu.edu/htc/Zasthma sim demo/ZapSimTempDemo.htm 

Evaluation 

Data that may be collected online from children is restricted, necessarily limiting the scope and detail of 
data that may be collected from the simulation. As the simulation completes, users are asked to indicate their age 
(eighteen and above or less than eighteen) and are directed to survey instruments developed for each age group. 
Attitudinal response to the simulation is queried, and recommendations for improvement are solicited. Additionally, 
the following data is collected for each visitor to the simulation website: 

• First name entry 

• Date and time entering website 

• Access to review mode and cumulative review time 

• Number of sections completed in testing mode, and pathway through the simulation 

• Score in testing mode 

This data will be analyzed to suggest refinements, and to ascertain average user time to complete the 
simulation. 

Lessons Learned 

Perhaps the most important, and surprising, lesson from this project is that computer skills and access in the 
Atlanta inner city environment for children is greater than anticipated by the project designers. Target learners in 
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this population were manifestly skilled in PC and Internet fundamentals, generally more so than their own parents, 
and adapt quickly and competently to the online simulation approach. 

Second, the branching -logic clinical simulation methodology appears to be a good match for this 
instructional goal and these learners. Representative learners related well to the virtual subjects and the content, and 
were able to effectively navigate through complex simulation sections, remediate errors, and earn respectable final 
scores. An unanticipated aspect of the simulation is the competitiveness that it generated among users, both children 
and adults. It may be preferable to report a general score category rather than exact numerical scores(McGuire, 
1973). 



The Future 

Metro Atlanta comprises a large population of adult asthmatics, and as in the case of children with asthma, 
adult blacks experience an incidence of the disease disproportionate to general population. Asthma management in 
adults is similar to that of children but treatment varies somewhat in drugs and dosages, and in lifestyle and 
environment factors. Anatomically, the small airway caliber of children increases the severity of asthma symptoms 
as the effects of airway inflammation and constriction are magnified(Farzan & Farzan, 1997). These factors cause an 
instructional program on asthma management for children to be relevant to adult asthmatics, but not a good fit. 
Therefore, a simulation targeting adult asthmatics in the same population is contemplated, with content and 
simulation approach and flow aligned with adult learner characteristics. For example, the role of smoking 
avoidance/cessation is more important in teaching adult asthmatics to manage the disease. Similarly, workplace 
triggers and stress are issues that should be emphasized in a simulation targeting adults. 

The growing Hispanic population in the U.S. presents another important learner population that would 
benefit from an online asthma simulation. Prevalence of asthma in Hispanic children is alarming and 
increasing(Centers for Disease Control and Prevention, 2004). A simulation for these learners will require careful 
attention to language, to accommodate various dialects of Spanish, and a selection of online characters that Hispanic 
children can relate to. There are important cultural differences in practices and attitudes toward medical intervention, 
especially in a disease with many myths and folk remedies, as is the case with asthma. 

Since inception of this project, Flash has become ubiquitous and available to mo st Internet users, and 
CourseBuilder/Javascript may be replaced with Flash to construct successive iterations of the simulation. 

Conclusion 

The simulation downloads and runs quickly even on older PC systems. It provides an engaging way to 
teach and practice asthma self management. We expect user data to confirm developer perceptions that the 
simulation website is effectively designed, and engaging and satisfying for target users. The website will be 
evaluated and refined in response to evaluation data and user comments, and as the subject content of asthma 
management evolves. This project will continue to serve a project-based learning role for Southern Polytechnic 
students. 
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Figure 1 Sample graphic (watching for asthma triggers) 
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Figure 2 Simulation cast, left to right: Jalesa (Michelle), Trellaine (Vanessa), Jamal (Jamal) and Bonita (narrator) 
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from 40,000 feet 





- maint meds 

- triggers 

- journal 

- action plan 

- contacts 

- resources 




- peak flow 

- "day" quality 

- self assessment 

- MD & clinic 



- rescue meds 

- see MD or clinic 
-911 orER 

- MD/clinic 



Figure 3 Simulation design overview 
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pathway 



Part 1: MANAGEMENT 



Part 2: MONITORING 



Part 3: ACTION 
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Figure 4 Simulation pathway (parallel sections assigned randomly, or are alternative/remedial sections) 
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Figure 5 Simulation storyboard (partial; PowerPoint) 
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Section F 



Type 

Checkbox 



N$, help Michelle/Jamal find the asthma 
triggers in his/her environment. Click all 
the triggers, then press <Done> 





1 


Hot or Cold 
Air 


% 


That's right. NS. many people with asthma are sensitive to 
air that is either hot or cold. 




1 


Smog 


is 


Yes. N$. smog is definitely an asthma trigger. 




1 


Exercise 


h 


Yes, N$. some people with asthma experience a worsening 
in their condition while they exercise. 




1 


Smoke 


<9 


You bet, N$! Smoking, or being around someone who is 
smoking, is a terrible asthma trigger. 




J 


Loud Music 




No, NS. loud music has not been linked to asthma. 



Done 



Figure 6 Simulation section from storyboard (PowerPoint) 
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Review Mode 



Section F 



Molly, help Jamal find the asthma triggers in 
his environment. 




Hint: More 
than one 
answer may 
be right 



Click 




(Click all the triggers, then click <GO>) Pl , 1 0 

# 



□ cold or hot air 

□ smog 

□ exercise 




□ smoke 




Sound On 

□ rock or rap music 




$ 



Click to 
show right 
answer(s) 



C'. 

^ GO 



Figure 7 Simulation section in distributed form 
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Online Asthma Simulation 
Certificate of Participation 



ZAP Asthma 



Date: Tue Oct 5 1 1:43:59 EDT 2004 
Name: Molly 
Score: 54 

Congra tula ti ons! 

Molly, you scored 54 points out of 61 possible 
points.You have earned the award of: 

Asthma Zapper FIRST Class 



This is the highest scoring category. 




Figure 8 Printable certificate of achievement 
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Igniting the SPARK: Supporting the Technology Needs of Online Learners 
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Abstract 

Students taking hybrid or online classes are often unprepared for the kinds of skills that are needed to be 
successful in this environment. This report provides an overview of one approach, an interactive CD-ROM 
(SPARK), that faculty can use to assist students in narrowing the gap between needed online learning skills and 
their current technical knowledge. 

The popularity of online learning continues to transform the educational landscape. As more faculty 
redesign courses to meet the demands of education in the 21 st Century, some students can be left behind. Students 
who have not used information technology in previous school experiences and those who are returning to school 
after a long hiatus from higher education are of particular concern . Even those students who consider themselves to 
be technically proficient may have developed bad habits over the years that create barriers for them in the online 
context. Faculty should recognize this potential “digital divide” and assure their students have the tools they need to 
be successful in online learning experiences. 

Online courses suffer from high attrition rates . A possible explanation is that students are not adequately 
prepared. According to Rowntree (1995), one of the key skills areas that students identify as requiring a “steep 
learning curve” for online learning includes computing skills (p. 212). The Student Preparation and Resource Kit 
(SPARK) was created to address gaps in knowledge between needed online learning skills and students’ knowledge 
deficits. SPARK has been piloted with two groups of nursing students: 19 undergraduates and 18 graduate students. 
Following is a description of SPARK, related definitions, a brief review of usability literature and a report of student 
evaluations of the CD-ROM. 



Description of SPARK 

SPARK was created through a partnership of the College of Nursing and MediaKube, LLC, a digital 
solutions provider and funded by the Arizona Regents University. The CD-ROM was planned to be easy to navigate, 
entertaining, and conversational. The decision to use this instructional style had two positive implications. First, 
students who considered themselves computer novices would be more likely to retain information presented in a 
non-threatening manner. Second, students who felt they already were familiar with the material would be enticed to 
explore the content for the entertainment value. 

A significant challenge was that the program had to effectively present items of a technical nature in a way 
that was not daunting for the user. Wherever possible, real-world analogies were used to relate terminology to 
something with which the student was likely to be familiar. For example, a flatbed scanner is compared to a 
traditional copy machine with the noted exception that the scanner output is sent to a computer via a digital signal 
instead of printed onto a piece of paper. Humor was injected throughout to make the content less intimidating and to 
facilitate the description of complex subjects. Remediation for wrong answers was provided in a helpful and friendly 
manner. The scripting allows students to repeat a question just to find out how the software reacts to the wrong 
answers. Learning why an answer is wrong can often be more educational than simply knowing the correct response. 

SPARK is an appealing visual experience with plenty of motion and imagery. This delivery style helps 
direct the immediate attention of the student, while at the same time giving them a mental image to recall at a later 
date when they need to apply the information they have learned. Where appropriate, animated simulations 
demonstrate the appropriate steps in a particular task prior to requiring the user to perform the task. 
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For ease of use, SPARK is configured to launch automatically when the CD is inserted into a PC. The 
navigation in SPARK is designed to be as unobtrusive as possible, while still providing a substantial degree of 
control for the student. The replay and skip buttons allow the student to quickly maneuver within a topic, while a 
click of the map button offers them a hierarchical view of the entire content tree. The student can navigate to any 
other program topic with just three or four clicks. 

SPARK Configuration and Navigation 

The program begins with an animated series of credits and title screens. The voice-over narrator starts by 
asking, “Is this the first time you’ve sat down to go through this CD or have we already met?’’. A click of button A 
“First time for me. ” takes the user through a full introductory sequence, while clicking button B “We’ve already 
met. ” directs them directly to the SPARK Topic Map. Similar branching occurs throughout much of the introductory 
section of the program for each main topic. The Topic Map displays the main categories of information followed by 
a layer of main topics. Below the main level is a set of sub-topics for each major category. The following table 
shows the overall layout of SPARK. 

Table 1. SPARK Category and Topic Layout 



Categories 


Main Topics 


Hardware 


Introduction, CPU, Memory, Storage, Input, Output, Connectivity 


Software 


Introduction, OS Software, Applications, Viruses 


Internet 


Networks, LAN vs. WAN, World Wide Web 


Skills 


Keyboard Shortcuts, File Formats, Using Adobe Reader, Using a Web Browser, Sending 
Email, Searching, Downloading, File Management 


Navigation Help 


A detailed explanation of each navigation button and feature is displayed on the Topic Map 
screen. 



Definitions and Usability Literature 

The following definitions are provided to clarify the meaning of various terms used in this study: 

1. Multimedia is the convergence of computers with motion, sound, graphics, and text (Azarmsa, 1996, p. 2). 

2. Hypertext is the presentation of information as a linked network of nodes which readers are free to navigate 
in a non-linear fashion (Keep, McLaughlin, & Parmar, 1993-2000). 

3. Hypermedia is a special case of hypertext that employs multimedia and describes linked information 
presentations that contain many forms of media (Azarmsa, 1996) that include sound, video, and so on 
(Keep et al., 1993-2000). 

4. Hyperlinks are the connections among units of information (nodes) in hypermedia. This arrangement can 
be described as a three-dimensional web of information (Dede & Palumbo, 1991, pp. 2-3). 

5. Computer literacy level refers to the ease with which a learner is able to operate the system controlling the 
hypermedia program. For example, a person with a low level of computer literacy may need assistance 
operating the mouse or keyboard commands necessary to navigate within the program 

Hypermedia Usability 

The term “hypermedia usability” refers to the ability to use a piece of hypermedia software for the intended 
audience. It pertains to the ease with which a learner can perform a specific search task for a particular piece of 
information. “Usability is the combination of fitness for purpose, ease of use, and ease of learning that makes a 
product effective” (Kushner, 2003). Usability has been applied to ‘the Web’ (the Internet) for a number of years; 
however, it is not specific to ‘the Web’. “Since the early 1980s. ...researchers have been investigating the usability 
and usefulness of hypermedia across a wide spectrum of domains” (Buckingham-Shum, 1996, pp. 1-2). 

Two main factors influence usability: content and design. Critchfield (1998) asserted that a well-designed 
website appears more credible regardless of the information provided. The usability of instructional multimedia 
(hypermedia) is vital for the success and satisfaction of its users because confusion resulting from poorly designed 
programs can be detrimental to learning performance. 

The process of assessing and evaluating online content is subjective and internal (Krug, 2000). Several 
approaches for expert -based evaluation of usability have been proposed over the past few years. According to 
Dimitrova, Sharp, and Wilson (2001) there is little evidence in the literature regarding the effectiveness of these 
approaches . Although expert evaluators are somewhat successful predicting usability problems, they still have 
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difficulties identifying certain types of learner problems such as comprehension. Expert evaluations do not eliminate 
the need for tests with actual learners. To that end, an evaluation by the end-user was deemed appropriate. 

Pilot Study and Evaluation 

SPARK was piloted at Arizona State University’s College of Nursing in the fall of 2004. The CD-ROM 
containing SPARK was distributed to nineteen members of an accelerated RN to BSN program and eighteen 
members of a graduate level neonatal nursing program All participants were allowed to keep the CD for their future 
use. Undergraduate participants received extra credit in their course; graduate students volunteered to complete the 
evaluation survey. The students were shown how to launch the CD in class and then asked to take it home to review 
it on their own time. They returned evaluation data via a seven-item survey (described below) the following week. 
Evaluation data were collected using a six item survey addressing level of confidence after viewing SPARK, its 
pace, ease of use, ability to keep participants’ attention, newness of material, and its usefulness. Participants ranked 
their responses to each of these questions on a five-point Likert scale ranging from “Strongly Disagree ” to 
“Strongly Agree”. A comment area was provided for each question. Finally, participants were asked what else 
should be included in SPARK as well as how long it took them to review the CD. 

Results of Evaluation 

An analysis of the data was used to determine what improvements and modifications should be made to the 
program. 100% of students from the undergraduate class and 51% from the graduate class responded to the survey. 
Means were calculated for responses to the Likert -type scale items; qualitative data were analyzed for themes . 

Table 2. SPARK Survey Items and Comparison of Means between Undergraduate and Graduate Students 



Item 


Undergraduate 

Mean 


Graduate 

Mean 


Overall 

Mean 


SPARK was easy to use. 


4.53 


4.89 


4.70 


The topics covered in SPARK were new to me. 


3.11 


2.50 


2.81 


The topics covered in SPARK were useful to me. 


4.11 


3.77 


3.95 


How (narration, self-paced units) topics were covered in 
SPARK kept my attention. 


3.84 


3.94 


3.89 


The pace in which topics were covered in SPARK was just 
right. 


3.63 


3.94 


3.78 


I feel more confident about my computer skills after using 
SPARK. 


3.58 


3.61 


3.59 


How much time did it take for you to review the materials of 
interest to you? (time in minutes). 


45.79 


29.64 


38.94 


Note. Undergraduate (n = 19), Graduate ( n = 18) 

Response scale ( 1 = Strongly Disagree, 5 = Strongly Agree ) 





Comments were analyzed for further insights into participants’ experience with SPARK. However, 
comments tended to mirror each groups’ rating of the evaluation items. Of the ten comments provided by graduate 
students, three students felt that the pace of the program was too slow to meet their needs and two students indicated 
that only some of the content was new to them. The undergraduate students provided many more comments ( n = 86) 
and were more positive in their evaluation. The two most frequent comments had to do with ease of use (« = 6) and 
enhancement of current knowledge (« = 6). Five comments indicated that not all of the content was new to the 
student. However, it appeared that SPARK was able to either reinforce information that students were unsure about 
or that it corrected misinformation. 

The amount of time spent in SPARK by undergraduate students as conpared to graduate students was 
significantly higher. Several circumstances may account for the difference. The undergraduate students were taking 
a class from one of the investigators (Hrabe); they also received extra credit for taking the time to complete an 
online survey. The graduate students completed a paper and pencil survey voluntarily (i.e., no extra credit) and the 
investigators were unknown to this group. The positive evaluations could also reflect participants’ gratitude for 
receiving a free copy of a CD and faculty concern for the students’ success in school. 
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Discussion and Summary 

Overall, data suggest a positive experience with SPARK. Ratings indicate that students’ felt the CD was 
easy to use, kept their attention and enhanced their confidence in learning the skills necessary to navigate online 
courses. While the lowest rankings indicated that much of the content was not new to the participants, having the 
information readily available helped to refresh and reinforce what they already knew and increased their confidence. 

Using SPARK or similar approaches highlights the importance of helping students acquire the technical 
expertise they need to be successful in hybrid or totally online courses. These endeavors should assist faculty in 
narrowing the gap between the skills students bring versus those they need. Future work will focus on improving 
assessment of skill and matching results to targeted remediation. 
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Table 3 . Selected Screens from SPARK, copyright and patent pending 2004. 



Figure 1. 

SPARK Title Screen 



Figure 2. 

Introductory screen asks user to rate his or her 
computer skills . Narrated voice-over feedback 
is individualized according to response. 



Figure 3. 

Program Navigation Instructions includes 
voice-over narration. 
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Abstract 

Compared to conventional classroom settings, e-learning relies heavily on a student’s reading ability. 
However, many students, particularly those at-risk or those who may have already dropped out of conventional 
schools, tend to have low reading ability that affects their ability to learn online. The problem is that relatively little 
has been done to address reading problems confronted by online distance learners and educators. E-learning often 
begins with an assumption that students can read. This study (a) identifies empirical-supported reading assessments 
employed by conventional schools, and (b) proposes reading assessment strategies for use by online educators. A 
review of reading assessment literature reveals that in conventional schools settings, classroom teachers are the 
primary people who detect students ’ potential reading problems; reading specialists are often called upon to further 
diagnose and treat reading problems; authentic assessments and reading software are being used as an integral 
part of classroom instruction to help students enhance their reading skill. The proposed assessment strategies 
include extant data analysis, learner-self and informant assessments, and reading-specific and performance-based 
assessments. 



Introduction 

Attracted by the potential of online distance education, many virtual schools are being established 
nationwide that are expanding curricula options for a wide range of students. Clark (2001) reported continued 
growth of virtual schools, with a trend from virtual high schools to virtual K-12 schools. Fourteen States had 
approved virtual schools and it was estimated that 40,000 to 50,000 K-12 students would take online courses in 
2001-2002 (Clark, 2001). 

The problem is that there is little empirical research to help guide distance educators and learners faced 
with reading difficulties. It is often assumed that students taking online courses and/or entering virtual schools can 
read. However, many students, particularly those at-risk or those who may have already dropped out of conventional 
school, tend to have low reading ability that affects their ability to learn online. While focus is being placed on 
address reading issues in conventional schools, there is a dearth of studies and information on how to address 
reading problems at a distance; when instructors and reading specialists are separated from the learner by geographic 
distance. Considering online courses rely heavily on a student’s reading ability, the potential problems are evident. 
Empirically supported methods are needed to assess and treat reading problems confronted by online distance 
learners and educators. 

Conventional K-12 schools are expending considerable time, effort and resources to enhance students’ 
reading skills and to meet state reading standards enacted by the No Child Left Behind (NCLB) Act of 2001 
(NCREL, 2002). The reading standards count for virtual schools as well, especially if they receive state -funding. In 
a virtual learning environment, however, it can be more difficult to monitor student progress than in the 
conventional setting: one reason is that, in the virtual environment, it is neither as easy to trace a student’s problems, 
nor to determine their causes (Easton, 2003). “There are no visual cues that suggest when a student is frustrated, 
confused, fired, or bored’’ in a virtual learning environment (Easton, 2003, p. 89). To the point, student reading 
ability remains significant to the progress of virtual schooling. 

It is apparent that if a student has reading problems, he or she may have difficulty in progress through 
online courses and may eventually dropout. Reading problems result from the deficit of cognitive development, 
comprehensive skills, or unfamiliarity to vocabulary or curriculum (Blachowicz, 1999). Online educators need 
reading assessment strategies to determine whether or not a low-achieving student has reading problems. Online 
educators also need a systemic approach to address such problems once they have been identified. 

The purpose of this study is to identify optimal reading assessment strategies for use by online educators 
across the curriculum. Specifically, this study (a) identifies empirical-supported reading assessment practices 
employed by conventional schools, and (b) proposes reading assessment framework for use by online educators. 
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Literature Review 

Four basic questions guided the review of literature. The findings were then used to formulate the reading 
assessment framework presented after the review. 

What are current reading assessment practices and how are they administered? 

The types of reading assessments put in practice vary from school to school. Denton (1999) reported that 
school districts employ national examinations, state-developed or state -recommended tests, and various informal 
measures to assess reading. However, if schools purchased assessment tools other than the state developed or 
recommended, they would not receive state -funding or reimbursement (Denton, 1999). 

The National Assessment of Educational Progress (NAEP) is administered once a year for students in 
grades 4, 8, and 12 to measure student progress in reading as well as other subject areas (NCES, 2003). The NAEP 
results are based on a sample of student populations, and NAEP does not provide individual scores for students or 
schools (NCES, 2004). 

Examples of state-developed tests include Georgia’s Basic Literacy Test (BLT) for K-5, Delaware’s State 
Testing Program for grades 34, Arkansas’ checklists for K4 and Louisiana Literacy Profile for K-l. State- 
recommended tests include, but are not limited to the verbal subset of the California Test of Basic Skills (CTBS) for 
K-l recommended for use in Maryland and a reading comprehension subset of the California Achievement Test 
(CAT) for grade 2recommended in Oregon (Baker & Smith, 2001; Denton, 1999), and Oklahoma’s Priority 
Academic Skills and Phonics Took Kit for grades 1-12 (Denton, 1999). 

State mandated reading assessments take place at least once a year, focusing on assessing a student’s early 
reading skills and monitoring their reading progress over time (Denton, 1999). Most school districts in the Southern 
Regional Education Board (SREB) states conduct early reading assessments at least once a school year. The 
majority of students range from Kindergarten through grade 3 but many districts extended reading assessments to 
grade 8 and even grade 12 to monitor students’ progress (Denton, 1999). The timing of reading assessments, 
however, does vary by state. For example, Tennessee school districts employed early reading assessment for 
prekindergarten through grade 3; South Carolina school districts employed it for grades 1-2; and Georgia school 
districts employed it for K-12 by modifying the levels of the assessment (Denton, 1999). Furthermore, NCLB Act 
requires that tests be administered every year in Grades 3 through 8 in math and reading in all schools beginning in 
the 2005-2006 school year (NCREL, 2002). 

Formal reading assessments are administered face-to-face by trained individuals in local schools either in 
one-on-one or small group settings. No instances of formal reading assessments were found in the literature that 
were administered at a distance. Furthermore, to ensure the coherence of assessment practices, the SREB states 
provide classroom teachers with training programs on administering the assessment (Denton, 1999). Although 
variance was found, the consensus among educators is that assessment practices should be balanced and coherent 
with instructional events to ensure a student’s acquisition of early reading skills; that the assessment employed 
should have evidence for reliability and validity based on research results, or alternative actions should be made to 
ensure the consistency of assessment practices (Denton, 1999). For example, Georgia Department of Education 
developed and implemented the Basic Literacy Test (BLT) for ongoing reading assessments, and used Iowa Test of 
Basic Skills (ITBS) to assess the effectiveness of BLT to ensure classroom teachers’ and students’ needs in reading 
(Denton, 1999) 

Table 1 depicts a current paradigm of reading assessment practiced in conventional schools. The current 
paradigm includes both formal and informal reading assessment practices described so far. Only a few examples of 
assessments are listed in the table based on the literature to help distinguish the categories. 
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Table 1. Current Reading Assessment Paradigm in Conventional Schools* 

Teacher-Generated Assessments 

- Direct: 

o Concepts About Print, phonemic awareness, phonics, & rubric -based assessments (Grades K-6) 
o Concepts About Print, phonemic awareness, phonics, literature response journals, & rubric -based 
assessments (Grades K-3) 

o Phonics awareness, running records, informal reading inventories, reading miscue analysis, qualitative 
reading inventories, & rubric -based assessments (Grades 4-6) 

- Indirect: Interviews, observations, analysis of writing samples or journals, teacher-crafted questionnaires, text 
discussions, etc. (All grades) 

District-Mandated or School-Mandated Assessments 

- Paper-Based Tests: 

o Arkansas’ Reading Recovery Programs (Grade 1 with low reading skills) 
o Georgia’s BLT: Basic Literacy Tests (Grades K-12) 
o Louisiana’s DRA: Developmental Reading Assessment (Grades 1-3) 
o Mississippi’s Reading Instructional Intervention Supplements (Grades K-8) 
o Oregon’s Oral Reading Fluency (Grades 1-3) 

o Virginia’s PALS: Phonological Awareness and Literacy Screening (Grades K-l) 

- Computer-Based Tests: 

o Accelerated Reader 5.0** (Grades not specified) 
o Autoskill Reading Program (Grades not specified) 
o Reading Mastery I & II*** (Grades 1-3) 

o Success -Maker Comprehensive Courseware System (Grades K-8) 
o SuccessMaker** (Grades K-8) 

o Yopp-Singer Test of Phonemic Segmentation*** (Kindergarten) 

State-Mandated Standardized Assessments: Administered onsite and at least once a year. 

- Delaware State Testing Program (Grades 3-4) 

- Georgia Kindergarten Assessment Program 

- Oklahoma’s use of the Iowa Tests of Basic Skills (Beginning in the grade 3) 

- OSA: Oregon State Assessment (Grade 3) 

- West Virginia’s Statewide Assessment Programs (Grades 1-12) 

National Standardized Assessments: Administered once a year. (Grades 4, 8, & 12) 

* indicates the table depicts only selected examples of assessments from the literature reviewed. 

** indicates the software as being assessment-specific tool. 

*** indicates the software as being intervention purpose. 

Teacher-generated assessments are divided into two groups: Direct and Indirect. Direct assessments are 
practiced by reading teachers for monitoring student reading progress, whereas the indirect assessments are typically 
administered by teachers to monitor student achievement of specified learning objectives (not necessarily reading). 
District- mandated or School-mandated assessments are depicted in two groups: Paper-based and Computer-based 
tests. Paper-based tests consist of state-developed assessments, while computer-based tests consist of reading 
instructional and assessment-specific software. State-mandated standardized assessments include state-developed or 
recommended tests. Finally, the national standardized assessments are shown with a little bit information about 
administration and target students. 

How are potential reading problems identified? 

Teachers are the primary people to identify and determine whether or not a student has potential reading 
problems. As Denton (1999) reported, teachers can use test results of either formal or informal assessment to 
monitor a student’s reading skills. The formal assessments include state -mandated standardized tests for monitoring 
reading progress, while the informal assessments include school-district or school-wide tests, curriculun>based tests, 
and teacher-strategic alternatives for ongoing performance assessment (Arthaud, Vasa & Steckelberg, 2000; 
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Campbell, 2001; Denton, 1999; Rueda & Garcia, 1994 & 2003). Curriculum-based reading tests include vocabulary 
tests, ‘end of unit basal tests,’ ‘skills continua’ and ‘end of the book basal tests’ to name a few (Arthaud, et ah, 2000; 
Rueda & Garcia, 1994). Teacher-strategic alternatives include, but are not limited to interviews with students, 
parents or previous teachers, ongoing observations, teacher-crafted questionnaires, text discussions, analyses of 
writing samples or journals, and monitoring of oral reading (Arthaud, et al., 2000; Rueda & Garcia, 1994 & 2003). 

Many states also provided assessment tools for teachers to use in identifying potential reading problems. 
Examples include, but are not limited to Arkansas’ Reading Recovery programs and Virginia’s Phonological 
Awareness and Literacy Screening (PALS) for potentially at-risk students (Denton, 1999). With increased access to 
computer technology, reading software applications also play an integral role in determining a student’s potential 
reading problems. Lor example, Thomas Intermediate School in TN used Accelerated Reader 5.0 to assess and track 
a student’s reading progress (Pittner & Coit, 2000). 

A teacher’s professional judgment is used to decide which assessments are appropriate to determine if a 
student has a potential reading problem. Variance in teachers’ choices is evident in the literature. Campbell (2001) 
found that the majority practicing teachers in selected K6 schools, regardless teaching experiences and grades, 
perceived four assessments to be highly effective: Concepts About Print (41%), phonemic awareness assessment 
(51%), phonics assessments (46%), and rubric -based assessments (44%). Teaching experiences and grades, 
however, revealed variance in the teachers’ choices; for example, beginning teachers (5 years or less experience) 
selected 3 assessments as being highly effective, while experienced teachers selected 1 1 assessments in the same 
category. K-3 teachers chose five assessments as being highly effective, whereas grades 46 teachers chose six 
assessments in the same category (Campbell, 2001). Similar differences were also evident among special education 
teachers and elementary bilingual teachers (Arthaud, et al., 2000; Rueda & Garcia, 1994 & 2003). Variance in the 
use of reading software confirmed that teachers established and used their own criteria as well as adopting policy- 
makers’ recommendations for selecting reading software (Byrd, 2001; Pittner & Coit, 2000; Sibenaller, 2001; 
“Texas Middle School,’’ 1995). 

A teacher’s role in determining a student’s potential reading problems is evident. Yet, it is anticipated that 
parents may be able to identity whether or not their child has potential reading problems, based on interactions with 
their child. In addition, parents who own reading software could use it for teaching their child and monitoring the 
child reading ability. 

If potential reading problems are found, what happens next? 

If a teacher thinks that reading problems may be considerable or are symptomatic of other problems, then 
the teacher typically refers the student to school psychologists or reading specialists for firther examination. 
Blachowicz (1999), for example, described how two children included in a multiple case study were referred to a 
reading specialist based on their teachers’ reports. Valleley, Evans and Allen’s case study (2002) described a seven- 
year-old boy, who was first identified as possibly having learning disabilities by his teacher, was referred to a school 
psychologist who determined that the boy had specific reading disabilities and was transferred from his public 
school to an outpatient center for treatment. 

Trained teachers or those who are assisted by trained reading professionals can diagnose a student with 
reading problems using formal reading/achievement assessment instruments, and the diagnostic tests are paper- 
based and administered onsite (L. Nelson, personal communication, October 8, 2004). The state of Elorida, for 
example, has selected three diagnostic assessment tools for school districts to use: those are ‘Fox in a Box,’ the 
'Diagnostic Assessment of Reading (DAR),’ and the ‘Early Reading Diagnostic Assessment (ERDA)’ (Torgesen, 
2004, p. 9). The Fox in a Box is designated to use by teachers who have trained or with advanced experience in 
administering the tests, whereas the DAR and ERDA are more appropriately used by reading coaches, school 
psychologists, or diagnosticians (Torgesen, 2004). As seen in Blachowicz’s and in Valleley, Evans and Allen’s case 
studies, school psychologists or reading specialists are the only professionals who diagnose students with learning 
disabilities and making decisions on follow-up treatments. Assessment software for use in diagnosing a student with 
learning disabilities has not yet been made available on the market (L. Nelson, personal communication, October 8, 
2004). 

If a teacher believes that potential reading problems are not too severe, then the teacher is typically the 
primary person to provide reading interventions. For instance, elementary teachers, including bilingual and special 
education teachers, employ various reading assessment tools to monitor student reading skills and use their results 
for making instructional decisions in reading (Arthaud, et al., 2000; Baker & Smith, 2001; Campbell, 2001; Rueda 
& Garcia, 1994 & 2003). Denton (1999) reported that many states provided school districts with intervention 
programs or supplements for helping students with low reading skills, such as Arkansas’ intensive reading programs 
during the school year and summer schools and Mississippi’s Reading Instructional Intervention Supplements. 
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Can software programs assess and improve reading skills? 

It appears that computer technology is making dramatic shifts in reading practices across the U.S. 
Conventional schools are adopting software programs, such as Autoskill Reading Program, Accelerated Reader 5.0, 
SuccessMaker, Success -Maker Comprehensive Courseware, and Reading Mastery to name a few, to help students 
progress in reading skills. School districts nationwide report reading progress and positive influence on schools’ 
reading environment using reading software in regular language arts courses and extra curricular reading activities 
with transient learners (Baker & Smith, 2001; “Texas Middle School,” 1995), students with low reading levels, the 
ESL, the gifted, the learning disabled (Pittner & Coit, 2000; “Texas Middle School,” 1995), students in remedial or 
supplemental programs (Baker & Smith, 2001; Byrd, 2001; Pittner & Coit, 2000), students in an intensive reading 
program or students with severe reading problems (Sibenaller, 2001). Pittner and Coit (2000) suggested that if 
decided to purchase reading software, educators should seek for advices from professional community and research 
results supporting the software, and should preview the software to seek its appropriateness for their students and 
subject areas. 



Proposed Framework 

A framework for monitoring and assessing reading skills in online learning environments is depicted in 
Figure 1 . The framework posits three interrelated steps that should be planned as an integral part of online teaching 
and learning. 




Figure 1. Proposed Reading Assessment Framework for Virtual Schools 

Table 2 depicts specific reading assessment that online educators may use during each phase of the 
proposed framework. The proposed assessments have been found to be useful to monitor and assess student reading 
skills and progress by published studies (e.g., Arthaud, et al., 2000; Baker & Smith, 2001; Byrd, 2001; Campbell, 
2001; Denton, 1999; Merchant, 2003; Pittner & Coit, 2000; Rueda & Garcia, 1994 & 2003; Sibenaller, 2001; “Texas 
Middle School,” 1995). 
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Table 2. Proposed Reading Assessment Strategies for Virtual Schools* 

Upon admission 
Extant Data Analysis 

- St ate -mandated standardized tests (Grades K-12) 

- District- mandated or School- mandated tests (Grades K- 12) 

- IEPs (Individual Education Plan) or permanent records (Grades K- 12) 

Beginning of the curriculum 
Learner-Self Assessment 

- Student interview (Grades K-12) 

- Publisher-provided tests (Grades K- 12) 

- CBTs (Computer-Based Test) (Grades K-12) 

- Teacher-crafted attitude surveys (Grades K-12) 

- State/School-district/School developed or recommended tests (Grades K- 12) 

Informant Assessment 

- Parent interview (All grades) 

- Teacher interview (All grades) 

During the curriculum 
Reading-Specific Assessment 

- CBTs (Grades PreK- 12) 

- Short answers or essays (Grades 1-12) 

- Writing products or journals (Grades K- 12) 

- Teacher-crafted CRTs (Criterion- Reference Test) (Grades K-12) 

- Modified forms of publisher-provided tests (Grades K-12) 

- Parallel forms of State/School-district/School tests (Grades K-12) 

Performance-Based Assessment 

- Short answers or essays (Grades 1-12) 

- Writing products (Grades K-12) 

- Teacher-crafted CRTs (Grades K-12) 

- Email discourse texts (Grades 1-12) 

- Virtual chat archives (Grades 1-12) 

- Discussion board posting texts (Grades 1-12) 

* indicates the table depicts recommended assessment instruments for online educators to identify a student’s 
potential reading problems across the curriculum. Student grades labeled consist of students ranging from regular 
to bilingual to special education students. 



Extant Data Analysis 

Extant data analysis consists of reviewing existing data related to a student’s reading abilities. Extant data 
analysis should be carried out upon admission by staff or online teachers. Materials for use in extant data analysis 
include state-mandated standardized test results, district-mandated or school- mandated test results, IEPs (Individual 
Education Plan) or permanent records. These materials can be useful for assessing reading abilities of students 
ranging from kindergarten through grade 12. Individual student’s test results and records may be entered into a 
database so that such extant data may be accessible to online teachers across the curriculum. Teachers may then use 
the extant data to facilitate student learning. Of course, such databases must be designed to protect students’ right to 
privacy. 

Standardized tests have been found to be most frequently used by special education teachers in 
conventional schools and play an important role in evaluating reading progress of students placed in special 
education and bilingual education programs (Arthaud, et al., 2000; Rueda & Garcia, 1994). Although formal 
assessments (aka., standardized tests) were not recommended for assessing individual student’s reading progress or 
for making instructional decisions (Arthaud, et al., 2000), formal assessments can be useful for online teachers to 
identify potential reading problems.. 
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IEPs and permanent records were found to be the most useful for assessing students’ reading abilities by 
special education teachers in conventional schools (Arthaud, et al., 2000). As all students have their permanent 
records and special education students have their IEPs in general, it makes sense that those materials can be useful 
for online teachers to determine a student’s potential reading problems and to make follow-up decisions prior to the 
beginning of their courses. 

Learner-Self and Informant Assessments 

Learner-self assessments ask student to assess their own reading abilities. Informant assessments ask 
someone who knows the student to rate the student’s reading abilities. Learner-self and informant assessments need 
to be done at the beginning of a curriculum by online teachers. As Guterman (2002, p. 284) noted, “One could not 
understand the children’s level of development, and therefore learning, without considering their actual development 
level and their potential development.” Successful educational practices for literacy growth of especially low- 
achieving minority students take into account sociocultural aspects, student individual experiences and potentials 
(Rueda & Garcia, 1994). The learner-self assessment is a way to get in individual student’s inner worlds, whereas 
the informant assessment to the student’s outer worlds that is also important to know the student’s contextual 
circumstances. 

Strategies or materials for use in learner-self assessments include student interview, publisher-provided 
tests, CBTs (Computer-Based Test), teacher-crafted attitude surveys, or state/school-district/school developed or 
recommended tests. Such strategies and materials can be useful for assessing students ranging from kindergarten 
through grade 12. Student interviews can be carried out via email exchange, postal mailing, or telephone. It is 
recommended that such assessments include open-ended questions. The usefulness of email exchange is backed by 
Merchant’s (2003) study whose participants were students ranging 7 through 10 in ages in writing projects in the 
U.K. The remaining assessment tools have been found to be most frequently used and useful for assessing student 
reading skills by practicing teachers in conventional schools (Arthaud, et al., 2000; Byrd, 2001; Campbell, 2001; 
Pittner & Coit, 2000; Rueda & Garcia, 1994 & 2003; Sibenaller, 2001; “Texas Middle School,” 1995). 

Strategies for use in informant assessments include parent or teacher interview. These interviews can be 
useful for gathering information about reading abilities of students in all grades. The interviews can be carried out 
via email exchange, postal mailing, or telephone. Usefulness of the informant interviews for student reading 
assessments has been identified by special education and bilingual education teachers in the literature (Arthaud, et 
al., 2000; Rueda & Garcia, 1994 & 2003). For example, a bilingual education teacher reported using student journals 
for communicating with parents and succeeding teachers about the students’ reading progress (Rueda & Garcia, 
2003). Therefore, the informant assessment as well as the learner-self assessment can be useful for online teachers to 
identify potential reading problems and to make follo\wup decisions at the beginning of a course. 

Reading -Specific and Performance-Based Assessments 

Reading-specific assessment stands for assessment administered by trained teachers, reading specialists and 
school psychologists that focus specifically on assessing students’ reading ability. Performance-based assessments 
represent instruments employed by teachers to measure learners’ achievement of learning outcomes in areas other 
than reading (e.g., regular course assignments and exams). 

Materials for reading-specific assessments include CBTs, short answers, essays, writing products, journals, 
teacher-crafted CRTs (Criterion- Reference Test), modified forms of publisher-provided tests, or parallel forms of 
State/School-district/School tests. The CBTs can be useful for assessing students in grades PreK-12. Reading 
instruction and assessment software have been in the market, aiming for Prekindergarten to grade 12 to adult 
learners, and they have been found to be useful for teachers to obtain immediate feedback on student progress and to 
make immediate follow-up decisions (Baker & Smith, 2001; Byrd, 2001; Pittner & Coit, 2000; Sibenaller, 2001; 
“Texas Middle School,” 1995). Short answers or essays can be useful for assessing students in grades 1-12. Because 
they are part of time-limited assessments in general, short answers or essays may not be useful for assessing PreK 
students in reading. 

Materials for performance-based assessments include short answers, essays, writing products, teacher- 
crafted CRTs, email discourse texts, virtual chat archives, or discussion board posting texts. Short answers or essays 
can be useful for assessing students in grades 1-12, but may not for PreK students because they are part of time- 
limited assessments in general. Writing products or teacher-crafted CRTs can be useful for assessing students in 
grades K-12. Teacher-crafted CRTs may include an alternative that says “I can’t understand what this question is 
asking for” in every quiz item to monitor student comprehension skills on a subject and to provide interventions 
thereafter. The remained assessment materials can be useful for assessing students in grades 1-12, but may not for 
PreK students because they require more sophisticated skills in Internet tools and delivery platforms. Usefulness of 
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and student age aptness for email exchange is backed by Merchant’s (2003) study as mentioned earlier. The virtual 
chat and discussion board as well as email exchange can be applicable for use in performance-based assessments if 
online courses are housed in a course management application that has such features. 

Summary 

The proposed reading assessment framework, including extant data analysis, learner-self and informant 
assessments, and reading-specific and performance-based assessments, should be planned as an integral part of 
online teaching and learning. The proposed framework is designed to help students maximize their potentials to 
learn online and to help teachers best serve their students in learning online. 

Teachers should become voracious consumers of extant data to determine if students have potential reading 
problems prior to their courses being started. Conducting learner-self and informant assessments, teachers should 
understand students’ inner and outer worlds to facilitate their learning at the beginning of a course. Yet, variance in 
students’ and informants’ competencies in communication tools should be taken into account for the assessments. 
Teachers should become consumers and generators of authentic reading assessment tools and strategies to facilitate 
student learning during the course. Furthermore, as Byrd (2001) and Sibenaller (2001) suggest, if reading software is 
selected, teachers should make sure it is appropriate for their students in their given learning environment and the 
teachers themselves should facilitate student learning. 

“Successful teachers obtain information about their students from high-quality assessments” (Nitko, 2004, 
p. 3). The proposed framework utilizes empiricaksupported reading assessments employed by conventional schools 
to identify potential reading problems. However, the effectiveness and efficiency of employing such methods and 
assessments online are still questionable. Further study is need to determine if conventional reading assessment 
methods are applicable online and if the proposed framework is suitable for guiding the implementation of such 
reading assessments to enhance online learning. 
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Abstract 

This study explored how presen’ice teachers perceived and utilized the Online Learning Support System 
designed and developed to provide a platform for presenting utilities like content, news, and course assignments, 
and facilitating communication among students and instructor. In this research, a qualitative method was used. The 
data was obtained through focus group interviews. Of 43 prospective teacher students enrolled in the “Instructional 
Planning and Evaluation ” course, offered by the Department of Educational Sciences at the Middle East Technical 
University, 15 students participated in this study. After utilization of the web site throughout the semester, students 
reported the advantages using online learning support system as: being independent concerning time and place, 
keeping contact with the instructor any time, accessing up-to-date announcements, and uploading the assignments 
in an electronic format. 



Introduction 

The swift development in computer technologies has increased higher expectations from educational 
institutions. Educational institutions are forced to prepare the next generation of citizens for the technologically 
oriented world. Teacher education institutions should educate prospective teachers who are able to create 
technologically enriched instructional settings and use advantages of technologies in instruction as well. It is 
universities’ responsibility to train prospective teachers with technology and how to apply technology to their 
instruction. In order for preservice teachers to achieve these intended outcomes, schools of education should provide 
appropriate training in using technology for instruction. According to The Office of Technology Assessment (OTA, 
1995) report “ The most direct and cost-effective way to educate teachers about technology is through the preservice 
education they receive in colleges of education or other institutions” (pp. 166- 167). 

Many faculty members have recognized the potential of using the Internet for instruction. The World Wide 
Web (Web) offers a great potential for education and provides variety of sources. The Web provides a user-friendly 
interface and easy access to text, graphics, audio, and video materials that may be used in a common and consistent 
format. Most educational web sites provide basic course information such as syllabus, schedule, announcements, 
and reading lists. Others go beyond static materials to include synchronous or asynchronous communication, online 
testing, discussion groups, conferences, whiteboards, streaming audio and video (Hazari & Schnorr, 1999). 

Exposing preservice teachers to using online supplementary material and after the course taking students’ 
perceptions on the utilization of the web site, factors affecting students web site usage rate and their suggestions on 
the web site could be helpful for the revised version of this tool. The findings of this study also could be beneficial 
while making decisions about alternative designs of the Web supported learning environments or tools for preservice 
teachers. 



Method 

Research Design 

This study explored how preservice teachers perceived and utilized the Online Learning Support System 
(OLSS); designed and developed to provide a platform for presenting utilities like content, news, and course 
assignments, and facilitating communication among students and instructor. The proposed study looked at the 
following questions: 

1. What are the students’ perceptions on factors affecting the usage of the web site? 

2. What are the students’ perceptions on usage of the utilities of the web site? 

3. What are the students’ suggestions about the effective use of the utilities of the web site? 
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In this research, a combination of qualitative methods was used in order to explore the research questions in 
detail. Due to the fact that very little researches have been conducted on how preservice teachers utilize online tools 
in their teacher training in Turkey, the initial data gathering needed to be more qualitative in order to gain a breadth 
view about the subject. 

Semi -structured interviews were conducted to gather in dept and broader information about the students’ 
perception of the web- supported course and course web site. Interview guides’ primary function is to help outline 
the topic to be investigated. Out of 15 students, who had indicated that they were willing to share their experiences 
with the online supplementary web site, three focus groups were constructed. The number of students each group 
contained was ranged from 4 to 6 students. 

Data Analysis 

The procedure conducted to analyze the qualitative data includes following steps: Data were organized, 
categories were generated, data were coded, and emergent understandings were tested. The procedure followed 
analyzing qualitative data was mainly centered on the procedures stated by Marshall and Rossman (2000) and Patton 
(1990). First, the researcher read through all the interview data to identify meaningful units based on the research 
questions and assigned descriptive codes to these units. For instance codes like “colorized text,” “summary,” 
“example,” “accessibility,” “testing” were used to describe data. Second, the descriptive codes that fit together 
meaningfully were grouped in categories such as “content presentation,” “self-assessment,” “usability,” and 
“support”. The categories enabled the researcher to identify the themes to present data. A third level testing 
emerging understanding was carried out to evaluate the whole data for their usefulness and patterns of themes. Later 
thematic coding was employed to generate the general themes that presented in there headings namely, “factors 
affecting the usage of the web site,” “usage of the utilities of the web site,” and “suggestions about the effective use 
of the utilities of the web site”. 



Findings 



Theme I: usage of the utilities of the web site 

In this study, students and the instructor utilized the online learning support system throughout the 
semester. Keeping contact with the instructor online, upload possibility and revising easily was stated by the 
students as the advantages of the web site. Some parts of the web site were used frequently and some parts were 
used rarely. “Announcement” part was emphasized as being one of the most frequently used tools of the web site. 
“Forum” part was seen as the least utilized part of the web site due to its insufficient facilitation. Since, lecture notes 
was composed of a heavy loaded content, which is not designed in an organized way, students did not preferred 
reading from the computer screen. Students used the “Help” part of the web site in order to learn how to use the 
various parts of the web site. 

The web site was not used for communication purposes frequently since students were having the 
opportunity to interact with each other and they prefer face-to-face communication instead of communicating via 
web. Although some of the students preferred to use the “forum” part for communication purposes, most of the 
students preferred to use telephone and GSM for communicating with each other due to immediacy. Possibility of 
face-to-face interaction and lack of privacy of the forum messages together resulted in some students being passive 
participants. “Forum” part was not also utilized effectively to create and carry on discussions. Some parts of the 
communications occurring between students and instructor were carried out by the “Forum” part of the web site. 

Theme II: factors affecting the usage of the web site 

Students found the download time of the web site fair enough for their work. Navigation of the web site 
was stated as being helpful enough to access all the parts of the web site. Visual elements provided within the web 
site were found to be making the web site easy to use. On the other hand, there are several factors that negatively 
affect the students’ usage of the web site. Introducing of the web site, accessibility, reading off the computer screen, 
facilitation, privacy, and up-to-date information on the web site were underlined by the students as affecting their 
usage of the web site. 

Lack of orientation toward the utilization of the tools was one of the major factors, which affected the 
utilization of the web site. The results revealed that students were not given enough information about the features 
and usage of the web site parts, which in turn underlies the importance of demonstration provided at the beginning 
of the course. The point, which got the students’ attention most, was the facilitation and administration of the web 
site. The guidance provided by instructor and keeping activities up-to-date played a key role for the frequent usage 
of web site. 



283 




Theme III: suggestions about the effective use of the utilities of the web site 

The suggestions made by the students in order to increase the utilization of the web site and to improve the 
effectiveness of the using web site as a supplementary for the traditional course mainly converged on the 
presentation of the content, usage of the communication tools, evaluation tools and facilitation of the web site. 

The results of this study showed that students enjoyed studying from PowerPoint presentations supplied 
within the web site. It was emphasized that the overall content on the web site should be comprised different types 
of resources and these resources should be presented in a summarized way, which causes the students to study from 
the web site. Students also underlined the lack of examples related with application in the web site. 

Most of the students proposed that the web site could be composed of assessment tools. They emphasized 
two points; one was utilizing the web site as a direct assessment tool, and the second was gathering feedback 
through the web site. Obviously, most of the students expressed their need for additional activities and questions 
regarding applications, as well as self-assessment. Most of the students emphasized that instructor should give 
immediate feedback when a question is posed to the web site. 

About the communication tools provided within the web site, students nearly converged on the same topics. 
They expressed their wish for communicating individually by using a private area supplied in the web site. It was 
commented that facilitation of students by actively using the web site is the key point in reaching effective usage of 
the site. Students were expected their instructor to participate to the discussions held in the forum more frequently. 

As a final point, some students stated that they were looking for a more enthusiastic and social environment 
within the web site. For creating such an environment, they suggested communication should not only be base on 
educational concerns but also social concerns related with contemporary issues and their role in the society. They 
also said that the climate provided in the web site should not be formal, but warm. 

Implications for Practice 

For further practice, the following suggestions may be helpful for instructors and instructional designers 
while designing, delivering and implementing a web-supported traditional environment. 

■ Provide demonstration at the beginning of the semester as well as providing continuous support and help 
throughout the semester. 

■ Provide a wide range of pre- and post activities (additional resources, additional materials, self-assessment 
utilities etc.) apart from the content. Providing self-assessment tools would help students to monitor their 
learning. 

■ Provide enough guidance and support to make students participate to the web site more regularly by posting 
questions to think on and discuss, sending examples, providing additional materials and Internet addresses to 
check for. 

■ Keep the designing of the web pages simple and consistent. Provide simple navigation by informing the 
student in which page he/she is at that moment and make all the utilities accessible in every page. 

■ Online supplementary tools should provide social environment to create online community. 

■ Online supplementary environment should carry tools that facilitate online communication and corporation 
between teacher and students among themselves. 

■ The site should support students’ privacy with providing feedback and grading system individually and also 
should provide private communication channels between students - students and students - Instructor. 

Conclusion 

This research sheds light on development and utilization of an online supplementary tool in preservice 
teacher education. The findings of this study assist other researches and developers in their development and 
utilization of web based systems. Moreover, preservice teacher educators can also benefit from the findings of this 
study. Based on this and similar studies’ findings, preservice teacher educators can revise and improve teacher 
credential courses. 

In order to create an effective use of web-based environment, the features of the web site should be used in 
a manner that parallel with the goal of the related utility. The key to effective web based/supported instruction is 
focusing on the needs of the students, the requirements of the content, and the constraints faced by the instructor. 

The overall content should be presented in various formats that meet different expectations of the students. Besides 
navigational support, the content should be kept in summary or outline form to minimize the reading from the 
computer screen. It is a widely known fact that in terms of visual design, each web page should be designed in a way 
that minimal information on each page is provided. Hence, together with these opportunities it is seen that adding 
self-assessment tools for students could play an important role for students’ motivation and interaction. As stated by 
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different authors, interactivity in a web-based environment is a key to success (Moore & Kearsley, 1996; Palloff & 
Pratt, 2001). 

Due to the fact that very little research has been conducted on how preservice teachers utilize online tools 
in their teacher training in Turkey, there is a continuous need for further research on the effective use of the Web in 
teaching and learning process for preservice teachers. Following researches should comprise more than one teacher 
credential course and participants from different subject areas of preservice teachers. In addition to taking students’ 
attitudes and perception on utilization of the online tools, teachers and administrators’ views and perception should 
also be examined. 



References 

Hazari, S. & Schnorr, D. (1999). Implementation of interactive web module in a teacher education course. Journal 
of Computing in Teacher Education, 75(3), 8-16. 

Marshall, C., & Rossman, G. B. (1999). Designing qualitative research. Thousand Oaks: Sage Publications. 
Moore, M. G., & Kearsley, G. (1996). Distance education: A systems view. USA: Wadsworth Publishing 
Company. 

Palloff, R. M., & Pratt, K (2001). Lessons from the cyberspace classroom: The realities of online teaching. 

San Francisco: Jossey-Bass 

Patton, M. Q. (1990). Qualitative evaluation and research methods. Newbury: Sage Publications. 

U. S Congress, Office of technology Assessment. (1995). Teacher and technology: Making the 
connection. Washington, DC: U.S Government Printing Office. 



285 




Building an Instructional Design Alumni Support Community: Tracking 
Alumni for Program Evaluation and Added Value 

Kathleen W. Ingram 
Linda L. Haynes 
Gayle V. Davidson- Shivers 
Richard Irvin 

University of South Alabama 



Abstract 

This paper is a report of the development of a systematic process for locating instructional design alumni 
and recruiting them to provide feedback from their workplaces. The purpose is to make alumni part of the process 
for program improvement. The process for tracking, contacting, and gathering data from alumni is described. In 
addition, the results from the survey are discussed. 

Introduction 

When a graduate program reaches a certain level of maturity, the need to systematically re-examine, re- 
evaluate, and reflect on the program’s success in preparing our alumni becomes very important. In addition, a 
significant group of alumni become potential advisors for continuous program improvement. 

The reasons for maintaining contact with alumni include networking for current students to obtain 
internships and professional positions, meeting requirements for accrediting agencies, and determining how well the 
program prepared the graduates for their professional careers. In order to strengthen graduates’ ability to move 
theory into practice, graduate programs need to stay current on professional practices. To determine what current 
practices are and to aid in evaluation and planning, documented information should be obtained from both alumni 
and professional organizations. 

The problem is how to maintain continuous contact with alumni and seek their advice regarding the 
program. Maintaining a sense of community among alumni of a graduate program is difficult. It is even more 
difficult if the program itself is made up of adult students who mostly work full-time, have family and other 
responsibilities, and perhaps do not have the time to invest in developing professional relationships while they are in 
the program. 

At least two important issues must be addressed in contacting alumni. The first is the need to develop 
instruments that will facilitate large response rates (Delaney, 1994; Underwood, Nault, & Ferguson, 1994). The 
second is locating current contact information for alumni when a complete database has not been developed. In fact, 
models for developing an alumni community are needed to assist with the process of contacting alumni and gaining 
important feedback to be used in current programs (Davidson-Shivers, Inpornjivit, & Sellers, 2003; Delaney, 1995). 

Based on the literature, our current level of data, and the needs of the program, we determined five broad 
questions that we needed to answer to understand the needs of our students and thereby improve our program. The 
research questions for this study were as follows: 

1. What are the career paths of Master’s and Ph.D. alumni? 

2. How well did the program prepare alumni for their career paths? 

3. At what level would the alumni like to be involved in the program? 

4. What can be done to make alumni participation/involvement easier? 

5. How can we keep a current database of our alumni? 

Methods 

In an effort to capitalize on previous efforts, our first step was to identify extant data sources within our 
organization. One important data source was a 26 -item Alumni survey administered in 2001 (Davidson-Shivers, et. 
ah, 2003). The 2003 study served as a foundation for developing the instruments for the current study. 

Other extant data, although collected for somewhat different purposes, were also identified. The data 
included the following; (1) online survey of current Instructional Design and Development (IDD) students 
(http://www.southalabama.edu/coe/bset/idd /surveys. html), (2) IDD student exit surveys, (3) Needs Assessment 
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report (Davis, Rivers, Conley, & Bowers, 2003), and (4) technical report of the perceptions of Ph.D. alumni of the 
program (Bates, Beck, McDonald, & Ward, 2003). Finally, the International Board of Standards for Training, 
Performance, and Instruction (IBSTPI) (Richey, Fields, & Foxon, 2000) professional competencies informed our 
survey design. 

In order to involve current students, we included the tracking system as a Needs Assessment (NA) project 
in one of our online graduate courses. The student team (Shaw & Irvin, 2004) was given the following “proble m” 
statement: 

This NA project is related to tracking Alumni of the Instructional Design and Development (IDD) program. 
Currently the IDD program does not have a formal system in place for tracking the career paths of our graduates. 

The Chair of the department has tasked the researchers with developing a process for tracking and contacting 
these alumni. Data are needed to determine where our graduates go, what competencies they need in their jobs, and 
whether or not they felt they were prepared by the program for these competencies. 

The students, Shaw and Irvin (2004), conducted the following data collection activities: (1) sent out 
information to both the ITForum listserv and our internal student group listserv to identify alumni contact 
information and (2) contacted faculty members within the program to determine what other data sources were 
available and what data sources needed to be explored. The outcome of their Needs Assessment was a well- 
developed tracking manual that included not only the process of setting up and maintaining a database but also the 
specific forms. In fact, many of the questions in the survey were based on Shaw and Irvin’s suggested survey 
questions. 

Based upon the literature (Underwood, et. al., 1994), we had originally intended to develop two separate 
surveys for Master’s and Ph.D. graduates to address the possible difference in career paths, knowledge and skill 
needs, and experience of each of the two populations. However, because many of the questions we had intended to 
ask had been answered in previous alumni surveys and because our research questions were somewhat broad, we 
determined that a single survey would serve our purposes. In other words, as we reviewed the extant data and began 
developing the survey we found that our original ideas for the survey design were not meeting the objectives of the 
research project. In addition, a number of graduates completed both the Master’ s and Ph.D. degrees in our program. 
Therefore, using a single survey was determined to be the best approach. 

The survey (Appendix 1) was created with an online software program. Survey Monkey. The survey 
consisted of six sections: (1) The Letter to IDD Alumni -- the request for participation, (2) Responses to Previous 
Questionnaires -- a checklist to identify participation in previous surveys, (3) Information about You — questions 
related to demographics and contact information, (4) Alumni Career Paths, (5) 1 st Southeastern Regional Conference 
for IDT (http://www.usouthal.edu/coe/srcidt) — information on an upcoming program conference and questions 
related to alumni interest in the conference, and (6) Anything Else — an open-ended item for additional comments. 
The questionnaire included a total of 23 questions. 

A list of alumni was obtained from the university Alumni office. A letter, asking the alumni to log on to the 
program website and complete the survey, was mailed to the 152 alumni on the list. A modified version of the same 
letter was posted to an international listserv (ITForum) for instructional design and technology professionals and to 
the program’s internal listserv. All three modes of communication included a request for the alumni to respond by a 
certain date. Because the response rate was only 23% by the given date, a postcard reminder was sent out to those 
who had not responded. The reminder generated additional responses and the final response rate was 30% (46 
respondents). 

Results 

The results of the online questionnaire are organized according to research questions. The related results of 
previous surveys (Bates, et. al., 2003; Davidson-Shivers, et. al., 2003) are included in some instances. 

Research Question 1: What are the career paths of Master’s and Ph.D. alumni? 

In the 2001 alumni survey (Davidson-Shivers, et. al., 2003), 18 of the Master’s alumni (78%) and 13 of the 
PhD alumni (81%) reported that they currently worked in a job related to ID. However, in the current study, only 16 
of the 33 Master’s respondents, or 48%, reported that they currently work in the field of Instructional Design and 
Development (IDD). Similarly, only 6 of the 13 PhD respondents, or 46%, reported that they currently work in the 
IDD field. The respondents who stated that they are not currently working in the IDD field provided the following 
information related to their current positions: 

• Student in IDD Ph.D. Program 
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• Assistant Professor 

• Managing computer labs and teaching microcomputing in education 

• I am presently working as an elementary school principal, so I'm kind of in the field, but maybe not for 
your purposes. 

• Instructor 

• Naval Helicopter Standardization Flight Instructor 

• Program Development Specialist 

• Retail Sales of Art and Fine Crafts 

• Gulfstream V Pilot for Dept of Homeland Security and US Coast Guard. 

• Doctoral student 

• I am the program coordinator (chair) of the Information Technology Program in the School of 
Computer and Information Science at [name omitted]. One of my duties is to develop new 
curriculum, I suppose that is related to IDD work? 

• Computer Programmer Analyst and Management Systems Specialist 

• Real Estate Investment. 

• I work in Human Performance Technology. 

• Since graduation, I have worked in the technology industry 

• Program Director [name omitted] Special Courses (education) 

• I'm currently an Assistant Prof, in the English Dept.; however, I am applying a lot of stuff (stats and 
research design) from IDD to my research in composition studies. 

• Administrative Support Specialist - Training Department 

• Mother 

• Financial Manager 

The alumni who responded that they are currently working in the IDD field supplied the following 
information about the title of their current position: 

• Senior Instructional Systems Specialist 

• Senior Instructional Designer 

• Instructional Designer - Coop Student 

• Instructional Design Specialist 

• Instructor 

• Assistant Professor 

• Associate Professor 

• Department Chair 

• Senior HR Manager/Curriculum Design 

• Instructional Designer and Online Course Developer 

• Multimedia Producer 

• Teacher 

• Technology Facilitator 

• Solution Planner 

• Training Coordinator 
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• Owner 



In response to the question “In what field do you currently work” the number of respondents by field are 
shown in Table 1 below. 

Table 1: Field by number of respondents and overall percentage (n=46) 



Field 


Number 


Percentage 


Higher Education Faculty 


13 


28.3% 


Business/Industry 


10 


21.7% 


Other 


6 


13% 


K-12 Faculty 


4 


8.7% 


Higher Education Administration 


4 


8.7% 


Military 


3 


6.5% 


K-12 Administration 


2 


4.3% 


Government 


2 


4.3% 


No Answer 


2 


4.3% 


Health Care 


0 


0 



Research Question 2: How well did the program prepare alumni for their career paths? 

Thirty-nine alumni (27 Master’s and 12 Ph.D.) described success stories connected to their graduate 
education. The predominant themes centered on the following areas: 

• application of theories 

• implementation of technology interventions in educational, military, health care, and corporate settings 

• design of courses in a variety of settings (e.g., higher education, adult continuing education, K- 12 
education, military training, and corporate training) 

• development of distance education programs in a variety of settings (e.g., higher education, adult 
continuing education, K-12 education, military training, and corporate training) 

• management of teams and large projects 

• human performance technology expertise 

• needs assessment expertise 

• evaluation expertise 

• recognition of expertise 

• recognition for excellent work 

The responses indicate the graduates do feel successful and are eager to attribute their success to their 
graduate education. The following quotes illustrate the success of the alumni. 

I rely on what I learned in the Ph.D. program every day as an assistant professor. I'm aware of and 
apply my ID skills when I design course materials and then deliver instruction. I have an extensive web-based 
support program for all of my classes, and it was designed based on my Ph.D. student experience. 

As a result of the combination of practical experience in a plant environment and information gained 
from 1DD I was able to enter a variety of corporate environments and contribute immediately. In a short 

period of time I was considered an expert in [company name omitted] Production Planning and Materials 
Management modules. 

Provide leadership, budgeting, scheduling, curriculum oversight and teacher evaluations related to 
Science classes (6-12). 

Just having the MS degree has done a lot. I get a lot of calls for work because of my experience and the 

degree. 
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I have slowly but surely convinced many non-profit organizations of the value of program 
evaluation and improvement. 

My IDD background and dissertation research have made me an evaluation 'expert' in the eyes of the 
different commands for which I have worked. All of the USA graduates enjoy a very good reputation within 

the Navy. 

Other indicators regarding alumni success and a positive graduate study experience were found in the open- 
ended questions asking for “Anything Else?” The positive opinions are shown in the following statements: 

I think USA 's IDD program is very strong. The professors stay current in IDD knowledge and 
practice! 

My IDD experience was very positive: Great instructors, great courses, great lab! Thank you all very 

much! 



The IDD program at USA is second to none! The curriculum is effective and the professors are 
outstanding. I enjoyed every course and learned so much during my time in the program. 

This degree is wonderfully versatile. I have been able to use my degree to stay employed in this 
hard economy. Students might have an interest in knowing how this degree can be tailored to 
different fields and applications. 

Additional positive comments were found among the 22 (17 Master’s, 5 Ph.D.) alumni who responded to the 
question regarding “challenge” stories. For example, one doctoral graduate offered the following statement that 
exemplifies responses from nine alumni: 

No challenge stories related to a possible lack of education from the IDD program. 1 feel very well 
prepared. 

Some challenges that were described were related to specific issues in the work environment. For example, 
references were made to getting stakeholders to identify what they really want and getting businesses to allow 
enough time for adequate needs assessment for training programs. Other challenges were related to keeping pace 
with changing trends in technology. For example, a Master’s graduate stated the following: 

Because my work is centered on developing computer based distance training, it would have been 
beneficial to include additional technology classes and a study of various delivery methods for distance 
education. 

The Survey of IDD Alumni for Academic Year 2001 (Davidson, et. ah, 2003) showed that alumni feel 
satisfied with their work and believe the degree helped them obtain their current job. Among the Master’s alumni, 
17 of 23 graduates indicated the degree helped them obtain their current job. Responses for the Ph.D. alumni 
showed that 1 1 of 17 graduates held the same belief. 

Research Question 3: At what level would the alumni like to be involved in the program? 

Part of the online questionnaire included a section regarding an upcoming program conference 
(http://www.usouthal.edu/coe/srcidt). The alumni were asked to suggest keynote speakers and possible topics for 
the conference. Alumni were also asked to indicate their interest in working on volunteer committees involved in 
the planning and organizing of the conference. A total of 17 (12 Master’s, 5 Ph.D.) alumni volunteered to work on 
conference committees. 

Research Question 4: What can be done to make alumni participation/involvement easier? 

The majority of alumni (83%) indicated the best way to contact them is through e-mail. When asked about 
preferred information sources for alumni, 61% indicated a preference for a web newsletter. Additional suggestions 
for information were offered by the alumni: 



290 




• IDD Resources (papers, how-to, etc.) 

• Educational offerings such as continuing education, workshops, jobs. 

• I would like to see the newsletter include a job opportun ity resource listing and conference 
information. It may also be interesting to know which conferences the IDD ( &/or BSET) dept is 
attending. 

Research Question 5: How can we keep a current database of our alumni? 

In a needs assessment for tracking IDD students, Shaw and Irvin (2004) developed a “Tracking Manual” 
that outlined a systematic process for creating and maintaining a database for tracking alumni. One of the 
requirements in the manual was a dedicated database administrator. 

An incentive for generating a current database of alumni was offered by one of the alumni respondents in 
the open-ended question asking for “Anything Else?” The graduate stated a desire for a database along with praise 
for the program and the faculty in this response: 

I would love a database of alums, and to be able to access a website where alums could interface and 
share experiences/ideas, and maybe recruit some talent! Again, the program gave me everything I needed to 
launch a successful consulting practice! I thank my instructors and professors for pushing us and bringing us 

the latest in ISD practices! 



Discussion 

Based upon our preliminary evaluation of the data, we believe that the continuing process of systematically 
collecting the data will increase alumni involvement in both the evaluation and future development of the program. 
The data from the surveys will be used for the following purposes: (1) determine the best tracking system, (2) 
determine the programs that will help build a better alumni community, (3) determine the best way to coordinate our 
alumni efforts within the university, and (4) determine what our program is doing right and what we could do better. 

An important consideration in maintaining contact with alumni was pointed out in two previous reports 
(Davidson et. ah, 2003; Shaw & Irvin, 2004). Administrative support and the allocation of human resources (e.g., 
graduate assistants or current student association officers) are imperative to the process of gathering and recording 
reliable alumni information. 

In terms of the best way to continue a relationship with alumni, the findings regarding modes of contact are 
important. The majority of alumni (83%) indicated the best way to contact them is through e-mail. The challenge 
then is to maintain current e-mail addresses. A potential way to do that is by requesting a permanent e-mail address 
that resides on the university’s server for all alumni - especially now that all current students at our university are 
provided an e-mail address for use in official university communications. In addition, a web newsletter and the 
potential opportunity to participate in a database for networking with other alumni may encourage graduates to 
submit current information about their experiences after graduation. The alumni interest in volunteering to work on 
committees for the upcoming conference indicates a genuine interest in affiliating with the program. 

The fact that at a recent program retreat one of the discussion threads was the creation and maintenance of 
an alumni database reinforces our belief in the need to institute an official IDD program database. From that 
discussion several key benefits of having a well -designed and updated alumni database were discussed. For 
example, past data have not been linked to demographics, which does not allow for a comparison of responses by 
respondent characteristics. In addition, the data need to be maintained systematically over time so that the program 
faculty can use it for longitudinal questions such as “Where are our current graduates going after graduation?” and 
“What are the competencies that our graduates are expected to have upon graduation?” Without a comparison over 
time, we have no empirical basis for program adjustments and improvements. 

Our findings show that the graduates of our program work in a variety of disciplines. One interesting 
finding related to alumni career paths was the number of graduates who stated they are not currently working in 
IDD. Yet, many of their job descriptions and titles lend themselves to applications of IDD (e.g.. Assistant Professor, 
Program Director, Human Performance Technology, and others). The next question then is to determine if we 
should ask the occupation question differently or if something is inherent in the way alumni define the program from 
which they graduated. 

While the results of the survey are context -specific, many programs in higher education are facing the same 
pressure to evaluate their practices (Davidson-Shivers, et. ah, 2003). The process and the results could serve as a 
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model for other ID graduate programs in their efforts to build community and evaluate their program effectiveness. 

The interest could be from a program evaluation viewpoint, an accreditation need, or strictly based on funding 

needs. Alumni tracking and the subsequent development of a strong community between alumni and the program 

will help connect academic theory and practice to successful contributions in society. 
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Appendix 1: Online Questionnaire 



Letter to IDD Alumni 



September 10, 2004 

The Instructional Design and Development program at the University of South Alabama is now in its 13th year and 
the time has come to reexamine the program. To better understand what changes are needed for today’s graduates, 
as well as build a stronger relationship with past graduates, we have developed an online questionnaire to gather 
information about the career path you followed as a result of your graduation from the Instructional Design and 
Development Program. The information will be used to evaluate the current IDD Ph.D. and Master’s degree 
programs and to determine what services our alumni need. All data will be reported in an aggregated form, therefore 
your individual information will remain confidential. 

If you have questions about your rights as a participant in this survey, please contact Ms. Pam Horner with the 
Institutional Review Board at the University of South Alabama (telephone 251-460-6308). If you have questions 
about this questionnaire, please contact us at the e-mail addresses or telephone number shown below. 

If you prefer to send your answers by mail or fax, please contact one of us and we will send a printed copy. 

We expect the questionnaire to take approximately 15 minutes to complete. Please reply by September 30th. 

Thank you very much for your assistance! 

Kathy W. Ingram, Ph.D. 

Assistant Professor 

e-mail: kingram@usouthal.edu/ 

Linda L. Haynes, Ph.D. 

Assistant Professor 

e-mail: Lhaynes@usouthal.edu 

Department of Behavioral Studies and Educational Technology 

College of Education, 3700 UCOM 

Mobile, AL 36688-0002 

Tel: 251-380-2861 

Fax: 251-380-2713 

Responses to Previous Questionnaires 

1. Over the past few years faculty and graduate students have collected information from current students 
and alumni. We are trying to build a database that takes all of these responses into account. In order to do 
that we need to know if you responded to previous questionnaires sent to IDD alumni. Please check all that 
apply. 

(2001) Questionnaire from Dr. Gayle Davidson-Shivers 

(2003) Questionnaire from students of Dr. George Uhlig (Bates, Beck, McDonald, & Ward) 

(2004) Questionnaire from students of Dr. Kathy Ingram (Irvin & Shaw) 

Information About You 

2. Please complete the following contact information fields to help us maintain current information. 

First Name 
Middle Initial 
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Last Name 
Suffix (Sr., Jr., etc.) 

Maiden Name 

Home Address Line 1 

Home Address Line 2 

City 

State 

Zip 

Business Address Line 1 

Business Address Line 2 

City 

State 

Zip 

e-Mail Address 

Home Telephone Number (please include area code) 

Business Telephone Number (please include area code) 

3. What is the best way to contact you? 

Surface Mail 

e-Mail 

Telephone 

Other (please specify) 

4. What information sources would you most like the IDD program to make available to you? Please check all 
that apply. 

Bi-Annual Newsletter 
Web Newsletter 
Alumni Web Site 
Other (please specify) 

5. Sex 
Male 
Female 

6. Ethnicity 

American Indian or Alaska Native 
Asian 

Black or African-American 
Hispanic or Latino 

Native Hawaiian or Other Pacific Islander 
White 

Other (please specify) 

7. Date of Graduation from IDD Master's Program (USA only) 

8. Date of Graduation from IDD Ph.D. Program (USA only) 

9. Please list any other degrees you received - including degree name, major field of study, institution/school 
name, and year --e.g.,B.A. English Florida State University 2001 

Degree 1 

Major 

School 

Year 

Degree 2 

Major 

School 
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Year 

Degree 3 

Major 

School 

Year 

Degree 4 

Major 

School 

Year 

Alumni Career Paths 

The following questions are related to your work. 

10. Are you currently working in the field of Instructional Design and Development? 

Yes 

No 

11. If you are not currently working in the field of Instructional Design and Development, please indicate 
your current profession. 

12. In what field do you currently work? 

K-12 Faculty 

K-12 Administration 

Higher Education Faculty 

Higher Education Administration 

Business/Industry 

Health Care 

Military 

Government 

Other (please specify) 

13. Where do work currently? 

Organization Name 

City 

State 

14. What is your job title? 

15. How long have you been employed in your current job? 

16. Briefly describe any other jobs you have had since your graduation from the IDD Program. Please include 
the company name, your job title, and a brief description of your duties. 

17. Please tell us about any success stories you have that link your work and your education from the IDD 
Program. 

18. Please tell us about any challenge stories you have that link your work and a possible lack q/ education 
from the IDD Program. 



1st Southeastern Conference in IDT 

The 1st Southeastern Conference in Instructional Design and Technology (IDT) will be hosted by the University of 
South Alabama Instructional Design and Development Program on March 1 1 - 13, 2005. The conference theme is 
Challenges of e -Learning and IDT. The conference was first conceived as one way to reconnect with our alumni. 



295 




We would like your input into the conference planning. Please answer the following questions regarding the regional 
conference to be held on March 1 1 - 13, 2005. 

19. Who do you suggest as keynote speakers for the conference? 

20. What topics do you suggest for the conference? 

Topic 1 
Topic 2 
Topic 3 
Topic 4 
Topic 5 

21. Please indicate your interest in serving on conference planning committees. Please check all that apply. 

Scheduling Committee 

Hotel and Travel Committee 

Program Design Committee 

Program Announcement and RFP Committee 

Proposal Review Committee 

Proceedings Committee 

Program Printing Committee 

Meals, Receptions, and Recreation Committee 

Conference Treasurer 

Registration Committee 

Materials and Supplies Committee 

Equipment Committee 

Conference Web Site Committee 

Exhibits Committee 

Sponsorship Committee 

Decorations and Signs Committee 

Hospitality Committee 

Conference Evaluation Committee 

22. Do you think you will be able to attend the conference in March 2005? 

Yes, definitely 
No, definitely not 
Maybe 

Anything Else? 

23. Please use the space below to tell us anything else you would like us to know. 

Thank You! 

Be sure to click the Done button when you are finished. 



Thank you for completing this questionnaire! 
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Simulations as Authentic Learning Strategies: 

Bridging the Gap Between Theory and Practice in Performance Technology 

Kathleen W. Ingram 
M. Katherine Jackson 
University of South Alabama 

Abstract 

This article describes the design, implementation , and evaluation of a diagnostic experiential simulation 
( Gredler, 2004 ) in a graduate Performance Technology ( PT) course. Simulations are experiences that provide an 
authentic learning environment that scaffolds novices' problem solving while minimizing the risks of ‘practicing’ 
their newly learned skills in a ‘real-world’ setting. The purpose of the study was to examine the effectiveness of 
using simulations with novice instructional designers and performance technologists as an instructional strategy for 
transfer of theory into practice. This study is a case study, Type one developmental research design (Richey, Klein, 
Nelson, 2004), which describes the design, development, and analysis of a simulation as a culminating instructional 
strategy and assessment. The student learning outcomes from the simulation are mixed, but overall the students 
perceived the simulation experience to be authentic and relevant to their learning. An unintended outcome was the 
development of a model for designing experiential simulations. 

Introduction 

It is well-documented (Conn & Gitong, 2004, Hybert, 2003, Medsker, Hunter, Stepich, 

Rowland, & Basnet, 1995) that in response to market needs and the subsequent maturation of the profession. 
Instructional Design and Technology (IDT) has transitioned over the past 25 years from focusing solely on 
promoting learning through well-designed instruction to a broader view of improving performance and 
organizational results. The expanded role of instructional design professionals requires an expanded set of 
competencies that include considerations for expertise in performance improvement. 

In response to the need for additional knowledge and skills, many instructional design programs now 
include Performance Technology (PT) skills training in their program. In fact Medsker, et al. (1995) found that out 
of the 82 programs they surveyed all of the HPT (Human Performance Technology) topics included in the survey 
were included in the curriculum to some degree. However, Medsker, et al. (1995) also found that out of the 18 
topics, training and training needs assessment were emphasized by more programs. 

PT, an applied discipline whose goal is to improve human performance in the workplace (Stolovitch & 
Keeps, 1999), involves the identification of multiple and possibly unique or hidden causes, such as administration 
and resources, for performance problems and the implementation of interventions to solve those problems 
(Stolovitch, Keeps, & Rodrigue, 1999, Guerra, 2003). Similar to ID, the PT profession has undergone changes in its 
title and definition in an effort to adequately describe the functions of the field. The evolutionary dynamics of PT 
and the changes incurred as a result have contributed to ambiguous professional competencies (Guerra, 2003). 

Using the ADDIE (analysis, design, development, implementation, and evaluation) activities (Gustafson & 
Branch, 2002) as the framework for her study, and comprehensively reviewing the literature related to PT, Guerra 
(2003) developed a list of competencies for performance technologists. These competencies were then validated by 
PT experts, who were asked to supply information about the frequency with which a competency should be applied 
and the frequency with which the competency is actually being applied. Although there were gaps between the 
frequency that was deemed optimal and the actual frequency of application, Guerra suggests that the discrepancy 
can be a function of the complexity of the decision making processes and influencing factors, such as organizational 
characteristics, inherent to PT. 

Stolovitch, Keeps, and Rodrigue (1999) also categorize PT skills in a classification similar to ADDIE. 
Through observations of PT professionals, Stolovitch et al. categorized PT skills into basic skills groups of 
requirements for analysis and observation; analysis; analysis and communication; design and communication; 
design; design, evaluation, and management; evaluation and management; design and evaluation; management; 
communication; and communication and interpersonal skills. 

Klein and Fox (2004) took a somewhat different approach in their study, where they assessed which PT 
competencies should be mastered by students in instructional design and technology programs. They surveyed PT 
practitioners and faculty from instructional design and technology programs, educational technology programs, and 
instructional systems programs. In the survey the participants were asked to rate the presented competencies 
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according to their importance for graduates entering the PT field. Klein and Fox (2004) found results indicating 
that competencies related to skills such as conducting performance and cause analyses and selecting and evaluating 
performance interventions were rated as more important than acquiring knowledge about PT models (p. 24)”. 

The literature on PT competencies (Guerra, 2003; Klein and Fox, 2004; Stolovitch, Keeps, & Rodrigue, 
1999) seems to reinforce the importance of providing authentic learning environments for PT novices by 
demonstrating the inter-related nature of PT competencies, including the need to develop general problem-solving 
skills. Even with knowledge and mastery of PT competencies, the PT technologist must exhibit expert decision 
making skills in order to fully assess the performance goals, gaps, and subsequent causes to appropriately apply a 
suitable intervention. 

Ill-structured problem solving is inherent to the field of PT. Domain -specific knowledge and skills are 
necessary to solve ill-structured problems, as well as highly proficient decision-making skills to address the 
ambiguities of the problem and its underlying causes (Ge & Land, 2003). Ill-defined problems are those that exhibit 
indefinite goals and offer opportunities for multiple solutions (Jonassen, 1999; Ormrod, 2004). 

Because of the ambiguity inherent in ill-defined problems, they require higher order thinking skills and 
allow for innovative solutions. Stepich, Ertmer, & Lane (2001) state that expert problem solvers are more likely to 
consider the problem’s foundational patterns and principles rather than simply its surface features. Sternberg (1998) 
suggests that an expert, along with the necessary informational knowledge, will possess the ability to form strong 
connections among well-organized schemata, filtering out irrelevant information (Patel, Glaser, & Arocha, 2000) in 
the process. Experts are also more inclined to consider the end results of the solutions on the organization or 
individuals (Stepich et ah, 2001). 

In order for knowledge to be applied in real world contexts, situated learning theorists suggest that learning 
take place in authentic contexts. However, Dick (1991) raised a concern about placing students in a complex 
situation, i.e. a constructivist-learning environment, for which many of them might not be ready. Ormrod (2004) 
suggests providing instruction in the necessary background knowledge for the context, facilitation of higher-order 
thinking, and providing diverse opportunities for gathering resources as well as opportunities for developing 
multiple solutions. 

Ormrod (2004) defines authentic activities as “tasks that are identical or similar to those that students will 
eventually encounter in the outside world” (p. 396). Driscoll (2000) identifies the focus of situated, or authentic, 
learning as the emersion of the learners in the culture of the field through interpersonal contexts, where they become 
apprentices and learn from experts in that field. Whitson (1997) suggests that situated learning supports more 
innovative approaches to problem solving and improved judgment and decision making when developing solutions. 

The use of authentic activities in the classroom allows the learners to recognize the interconnectedness 
among concepts with consideration for context and therefore, facilitates transfer to alternative situations and settings 
(Ormrod, 2004). Situated learning allows the novice to recognize and experience the relationships and interactions 
among people, contexts, and cultures as they influence and shape the knowledge required for practical application 
(Agre, 1997). 

However, a criticism of situated learning (Ormrod, 2004) is that there is a possibility of reduction in 
transfer to alternate settings that differ significantly from the original environment in which learning occurred. It is 
important that learners not only recognize the relevance of the knowledge in other contexts (Ormrod, 2004) but also 
learn how to use the knowledge they have acquired (Schank, Berman, & Macphearson, 1999). Specifically, learners 
must learn how to use and transfer the knowledge they have acquired to new contexts (Mayer & Wittrock, 1996; 
Schank, Berman, & Macphearson, 1999). 

Simulations have been used in educational settings for well over three decades (Dickinson & Faria, 1997; 
Gredler, 2004) to provide authentic learning environments. However, while there is an extensive literature base on 
simulations and games that spans many different disciplines, there is no generally accepted typology (Wolfe & 
Crookall, 1998). 

Gredler (2004) breaks simulations into two types, experiential and symbolic, primarily based on whether 
the learners’ role is external or internal to the simulation. In an experiential simulation the learners’ role is as an 
internal participant. Gredler further differentiates experiential simulations based on the types of contingencies that 
the simulation provides. A diagnostic, experiential simulation’s contingencies are “based on the optimal, near- 
optimal, and dangerous decisions that may be made” (Gredler, 2004, p 574). 

Because simulations are “open-ended evolving situations with many interacting variables” (Gredler, 2004 
p. 571) they can provide a model of a real world environment that contains ill-defined problems requiring analysis of 
many variables and may require more than one course of action. The ill-defined nature of the problems posed in 
simulations provides an authentic learning environment for PT novices. 
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While simulations may not provide an exact replica of a performance environment, perhaps limiting 
transfer of learning, they do offer many advantages over other instructional strategies. Because simulations are 
scaled down models of actual performance environments, they are able to focus on key issues related to the course 
content. This allows the novice to focus on a limited, but important, number of variables. Many authentic learning 
environments, in an attempt to increase the fidelity of the learning and consequently transfer of training, place the 
novice practitioner in an actual workplace setting. Simulations, on the other hand, offer a safe environment in which 
to practice their decision-making skills. 



An Applied Research Approach 

As in most professional disciplines. Instructional design is a blend of theory and practice. One strong 
research methodology that recognizes the importance of empirically examining instructional design practice is 
developmental research. Richey, Klein, and Nelson (2004) describe two types of developmental research. Type one 
and Type two. Type One developmental research “involves situations in which the product development process 
used in a particular situation is described and analyzed and the final product is evaluated” (p. 1 102). The purpose of 
this type of research is to produce context -specific knowledge that provides empirical evidence that serves a 
problem-solving function (Richey, Klein, and Nelson, 2004). The purpose of Type Two developmental research is to 
examine the general processes of instructional design, development and evaluation. 

The context -specific nature of the types of research questions posed when designing and implementing a 
simulation fit well within Type One developmental research. The questions this research project investigated were 
related to the types of learning outcomes, both cognitive and motivational that are promoted through the use of a 
diagnostic experiential learning simulation. More specifically, the two questions that guided the design of the 
course, the simulation, and research methods are 

o Will an extended diagnostic experiential PT simulation foster the learning outcomes identified from the 
literature on PT competencies? 

o What are the necessary components of a PT simulation for a) promoting & evaluation discipline specific 
learning outcomes and b) ensuring content validity, as well as content/context fidelity. 

Course Design 

Similar to other ID programs (Medsker, et ah, 1995) the curriculum for the Instructional Design and 
Development Masters and PhD programs at our university include topics related to PT. In fact the Masters program 
offers a concentration in Performance Systems and Training. However, this is somewhat a misnomer in that there is 
only one course, Performance Systems Technology, whose focus is entirely PT. Therefore, the course serves as both 
a survey and an application course. 

From a content analysis of the PT competencies discussed in Guerra (2003), Stolovitch, Keeps, and 
Rodrigue (1999), and Klien and Fox (2004), we divided the PT competencies into seven general categories. The first 
two categories (Conceptual Framework and Planning) were related to skills, knowledge and attitudes (SKA) that 
were global rather than domain specific. For each of the seven categories we developed core objectives. The other 
five categories were sorted into the ADDIE activities used in earlier categorizations, and because it was a familiar 
model for our learners. The following matrix of objectives by category guided our course design. 

Table One: Course objectives derived from the PT competency literature (Guerra, 2003; Stolovitch, Keeps, and 
Rodrigue, 1999; and Klien and Fox , 2004) 

Conceptual Framework: 



1. Define performance technology and its philosophical, systemic, theoretical and organizational underpinnings. 

2. Describe the role of the Performance Technologist within an organization. 

3. Identify and describe the fundamental components of an HPT model (the systematic combination of 
performance analysis [PA], casual analysis [CA], and Interventions). 

4. Compare and contrast HPT models. 

5. Describe a variety of specific performance technology problems. 

Planning: 

6. Apply fundamental research skills to a PT analysis. 

7. Determine the organization’s cultural climate for change. 



299 





8. Identify change theory approach. 
Analysis: 



9. Distinguish between performance problems requiring instructional solutions and those requiring non- 
instructional solutions. 

10. Conduct a performance analysis for a specific situation to identify how and where performance needs to change 
(performance gap). 

11. Determine the importance of gaps between what is and what should be at all organizational levels. 

12. Conduct a cause analysis for a specific situation to identify factors that contribute to the performance gap. 

Design: 

13. Match improvement efforts to organizational mission and strategy. (Context specific) 

14. Develop recommendations concerning: 

a. What must be improved to maintain required performance? 

b. What must be maintained to improve performance? 

c. What must be abandoned to improve performance? 

15. Apply change theory in your approach to PT. 

16. Apply systematic research-based design principles. 

17. Identify and prioritize possible training and non-training interventions to performance problems at all levels. 

Development: 



18. Determine resources, time, money, people, and existing interventions. 

19. Develop resources specs. 

20. Produce required performance intervention according to design specs. 

Implementation: 



21. Derive implementation plan based on intervention requirements. 

22. Derive implementation plan based on organizational dynamic. 

Evaluation: 



23. Plan a formative evaluation, based on pre-specific performance objectives: 

Communication : 



24. Communicate effectively in visual, oral and written form. 

25. Demonstrate appropriate interpersonal, group-process, and consulting behaviors. 

While the learners in the PT course consisted of adults, they were novices in the area of performance 
technology. Therefore, to prepare the learners for emersion in an authentic learning environment we implemented a 
combination of instructional strategies early in the course. These instructional strategies provided the necessary 
conceptual basis (entry level skills) needed to solve the ill-defined problems the learners would be faced with in the 
simulation. In addition to classroom discussions of the text material, the early instructional strategies included case 
studies, PT model building, and the design and discussion of several non-instructional interventions. 

[See Figure One] 

Simulation Design 

While this simulation was implemented the previous year in the same class, the simulation was still in the 
formative evaluation stages. The simulation for the current PT course was revised based upon the objectives derived 
from the PT competency research, an analysis of the student end-of course surveys and the audio -taped simulation 
debriefing from the earlier course, and a systematic model of simulation development that emerged during planning. 

As stated earlier, the course was designed around specific core objectives related to competencies needed 
for successful PT practitioners. St. Germain and Laveault (1997) state that because the benefits of simulations are 
“mostly qualitative in nature, it is important that the learning objectives and the means for evaluating them be well 
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established” (p. 321). Our instructional analysis supported St Germain and Laveult’s claim, in that we determined 
the simulation design process needed to be more systematic to ensure that the simulation met the learning objectives. 

The data from the earlier course, and a more extensive review of the simulation literature, led us to 
conclude that we also needed to address the content validity of the simulation (Dickinson & Faria, 1997; Feinstein & 
Cannon, 2002; St-Germain & Laveault, 1997), and to improve the overall fidelity of the simulation. In order to 
address these important design considerations and to move the process in a more systematic direction we developed 
systematic design model for designing and developing diagnostic experiential simulations. 

[See Figure Two] 

In an effort to increase the content validity of the simulation a PT expert evaluated the simulation materials 
before they were implemented. The materials were presented in a workbook that included an overview of the 
performance technology problem, the background information related to the simulation, the scripts for each actor, 
the data available to the simulation participants, and a process map of the simulation. The workbook moved the 
expert through the simulation in roughly the same steps the students would proceed, only with more in-depth 
information. 

After each step, which represented a timeframe in which the students gathered data through actor interview, 
the experts were prompted in an ‘‘Expert Notes” page to ask questions they might ask during that step of the PT 
process. The process was designed to help the expert reflect on both the performance analysis process, explicitly, 
and on the fidelity of the simulation. For example, in the notes from the last “interview” section she was asked to 
answer two questions: What ideas do you currently have based on the information you’ve received, and Would you 
ask additional questions, or follow-up questions? And in the second section of the expert notes sheet the expert was 
asked What questions do you still have, and How will you answer the questions? (given the avenues of data 
collection you’ve been afforded.)? In the third, and final section the expert was asked to list any questions or 
comments she had about the content of the simulation. 

Based upon the expert’s feedback several changes were made to increase content validity and simulation 
fidelity. For example, many details were added to the simulation in the form of actor’s notes and available data 
based on the PT expert’s statement that “Data about the problem is not sufficiently specific” and that the data 
“Seems vague on metrics or benchmarks, both internal and external”. After the simulation had undergone 
modifications based on the expert’s recommendations the simulation was implemented in the second half of the 
semester during a six period. 



Simulation Implementation 

The simulation was implemented in a graduate level Performance Technology (PT) survey course within an 
Instructional design program. Because the course is only required for Masters students in one of the three 
concentration areas offered in the program and is offered online in the fall and on campus in the spring, the course 
enrollment tends to be low. In the Spring of 2004 there were three Ph.D. students and three Masters students 
enrolled in the course, but only five completed the course. 

On the first night of class each student filled out a 36-item survey that consisted of questions about 
demographics, general knowledge skills, and performance technology competencies. The first four questions were 
demographic questions. There were seven questions related to general knowledge areas, e.g. word -processing, use of 
on-line resources, communication, team skills, and metacognition. The remaining 25 questions corresponded to one 
of the PT competencies listed in Table 1. 

The students then participated in a short simulation in which they were confronted with a performance 
technology problem. The students were divided into dyads, presented with a scenario that included the performance 
problem and the current data. Each group was asked “What are your next steps (i.e. what do you need to know and 
how will you get the information)?” Their decision-making process and solutions were intended to provide a base- 
line measure for comparison to their final project solutions and PT processes. 

Because the PT course was a survey course, there was a great deal of content to cover. In the first half of 
the sixteen-week semester the students were required to read and discuss the fundamentals of performance 
technology. These instructional strategies were designed to move the learners from a novice level to one of adaptive 
expertise (Bransford, et. al, 2000). Fore example, the students were assigned several case studies to analyze, in 
writing, and discuss in class. These case studies built upon the foundational concepts from the readings, and in turn 
provided an opportunity to practice the problem-solving skills required to move from needs assessment to problem 
identification in a PT systematic process. The problems in the case studies, and their related interventions, were 
similar to the four performance interventions the learners designed throughout the course. 
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The second half of the semester, weeks nine through 15, was dedicated to the simulation experience. The 
simulation began by providing the students with background information on a United States -based animal health 
care company, the perceived performance problem to be addressed in the PT analysis, and the key stakeholders 
involved in the project. As stated earlier, this was the same simulation used in the same course the previous year, 
only modified based on student and expert feedback. 

The initial information was provided to the two “performance technology teams” by their employer (the 
first author and instructor of the course) in preparation for an interview with the animal health care company’s “VP 
of Human Resources” (the second author). The following is a excerpt from the initial information provided to 
students: 

You and your team members have been recently hired by a 
small Human Performance consulting firm, PerformING 
Solutions. You will be located in the home office in (name of 
University Town). Although in your previous job your main 
responsibilities involved large instructional design projects, 
you have some experience with small Performance 
Technology projects. In fact, each of you has had some 
success with the implementation of the interventions that you 
suggested in your last position. Because of your part in the 
success of these projects, the CEO of PerformING Solutions, 

( 1 st author’s name) felt that you were well qualified, as a team, 
to conduct a performance analysis of the gaps and 
opportunities related to veterinary vaccine sales in North 
America for a new Client, ( Company’s name) 

During the initial interview the two teams were also provided with an interview schedule with upper and 
middle management in the form of a flight itinerary. 

Both teams were given the same information about the company and the interview processes in the first 
meeting. They were also told that everyone had the same opportunity to ask for additional interviews if they felt they 
needed additional information. The only caveat was that they email their requests to their employer (the instructor), 
rather than make the requests in class. The intent of separating the two groups’ process was to see what differences 
in the PT processes would develop from the choices the two teams made about what data to collect. We 
hypothesized that the questions each team asked in the interviews would influence their decision-making process 
and their choices about which additional data to pursue. 

During the simulation the two student PT teams interviewed actors recruited from the instructional design 
program, the drama department, and the management department at our university. While the first meeting with the 
VP of Human Resources (the 2 nd author) took place with both teams, all other interviews were conducted twice- 
once with each team. The actors were given a copy of the same information on the animal health care company that 
the students received. In addition they received written and verbal information about their individual roles, their 
relationships with their “colleagues” (actors) and an overview of the processes and expectations for the interviews. 

All interviews were recorded in hopes that the decision paths, as evidenced by their questions, for the two 
teams could be compared. In addition, each team met with the instructor halfway through the simulation to discuss 
their processes, i.e. within their chosen PT model, and their current ideas for causes, gaps, and interventions. This 
discussion, in the form of an interim report, was designed to make the teams’ implicit decisions explicit. In addition, 
the discussion of the interim report with the team’s employer (the instructor) was an effort to add fidelity to the 
simulation. 



Simulation Evaluation and Results 

The final product for the course was a PT analysis report and presentation. The reports were analyzed and 
compared to both the pre -simulation outcomes and the decisions made during the actual simulation. In addition, the 
two groups participated in a whole -class debriefing, which was also audio -taped. As part of the debriefing the 
participants completed two surveys, a post-course/simulation satisfaction survey and a retrospective self-assessment 
of PT competencies. 

In the pre -simulation each of the three pairs of students moved quickly to drawing conclusions and 
suggesting interventions. Neither group discussed putting into place a process for next steps or determining what 
data they currently had and what data they still needed. 
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A paired sample t test was conducted to determine if there was a significant difference between the 
Retrospective Self Assessment of Competencies and the Final Self Assessment of Competencies for the class as a 
whole (with a rating scale of 1 - 5; 1 = low and 5 = high). Results indicate that the average rating for retrospective 
competencies (M = 1.56) was significantly lower that the average rating of the final competencies (M = 3.92), ?(24) 

= -18.504, p < .01. This suggests that the students felt they had significantly increased their mastery of PT 
competencies after completion of the simulation and the course. 

Flowever, from an analysis of the final product it was fairly evident that while the students felt that they had 
increased their PT skills overall, there were still some competencies that needed to be improved. For example, one 
group did not go beyond the data that was given to them during interviews or provided by the instructor. This was 
indicative of a poor inquiry process and an overall lack of applying research methodology principles. While the 
other group did a great job of going beyond the data, their presentation of the information to the stakeholders was 
not well organized and was written with a lack of sensitivity for the stakeholders. This was an indication of their 
lack of communication skills and, to a lesser extent, a lack of application of change management principles. 

While one group did discuss the PT model that they used to guide their processes, neither group included a 
graphical representation of a PT model or used the steps of a model to guide their learners through the document. 

This could be interpreted as either an outcome from the end of semester rush to completion or a more substantial 
lack of understanding of how to apply a PT model. Because neither group chose to include a model, a third 
explanation could be a misunderstanding of the assignment. In any case, the fact they chose not to use a model 
showed a lower level understanding of how to organize and report data to stakeholders. 

However, compared to the pre -simulation outcomes the final products for both teams showed vast 
improvements. Both teams identified the gaps and the causes for those gaps and then aligned their solutions with 
those causes. The team that sought out additional data did a better job of providing a more holistic approach to the 
performance problem by suggesting more global interventions. For example the group’s suggested interventions 
addressed the following areas of concern: Organizational Communication for all both internal and external 
customers. Job Analysis/Work Design, Organizational design and development, and Organizational values in 
addition to training interventions. However, focused on more of an individual level, i.e. for the employees that they 
interviewed. The second team’s suggestions were for Soft skills training. Job analysis and work design. Motivational 
system. Organizational communication- between internal clients, and Benchmarking for the two groups they 
interviewed. In addition, the data from the final project showed that the team that provided a more holistic view of 
the problem also tied their suggestions to their data, whereas the other team’s suggestions were not tied to data, 
rather they appeared to be based on opinion. 

After each of the two teams presented their final project to the stakeholders (actors and instructor), the first 
author conducted a debriefing. Immediately after the debriefing, each student completed an individual questionnaire 
about the course, which included questions related to the entire course and specifically to the simulation. The survey 
was divided into three sections, General Course Evaluation, Course Topics & Assignments, and Simulation 
experience. One of the questions asked the students to rank the three most relevant assignments in the class. The 
choices were case studies, intervention assignments, simulation exp erience, simulation report, group charter, and 
individual project. All five respondents chose Case studies as one of the three most relevant assignments, whereas 
only four of the respondents chose the Simulation experience as one of the three most relevant assignments. 

Similarly, the responses to Simulation experience questions were mixed. When asked to choose what level, 
high, medium or low, best represented the fidelity of the simulation two respondents chose high, two chose medium 
and one chose low. However all but one of the students chose the highest rating for the question about how effective 
the debriefing of the simulation was for their learning. 

Conclusions 

There were several research methodology problems with this study. First, the two groups were not matched 
on their general skills or knowledge and skills related to PT. This makes any comparison across groups invalid. 
However, it did provide an interesting look at each group’s decision-making process. 

A second methodological problem was the incompleteness of the data. While we tape-recorded all of the 
interviews, several of the recordings were lost. Our original purpose in recording the interviews was to compare 
each group’s decision paths based on the questions they asked as they progressed through the simulation. Because of 
the lost recordings we have no such comparison. 

The final methodological problem was that of emerging data. As stated earlier, in the pre -simulation each 
group seemed to jump to interventions rather than determine the next steps in data gathering. If we had analyzed our 
data as it emerged, rather than at the end of the study, perhaps we could have helped the teams develop better data 
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gathering skills. Again, perhaps this too was an interesting outcome in that we discovered that our students don’t 
have a high level of research skills. This is an important PT competency and therefore a skill that needs to be taught 
or reviewed. 

Next we’ll specifically address our two research questions. There were mixed results for the first question, 
Will an extended diagnostic experiential PT simulation foster the learning outcomes identified from the literature on 
PT competencies. The students’ self reports did show that they felt that they had a significant increase in the 13 PT 
competencies in Table one. However, their final projects showed that they were still weak in several general 
competencies, i.e. competency number 6 and 16: applying research skills, and several PT competencies, i.e. 
competency number 15 (applying change theory), 17 (identify and prioritize interventions at all levels), 24 and 25 
(communication skills). 

St-Germain and Laveault (1997) state that because of the qualitative nature of the learning outcomes from 
simulations both the learning outcomes and the evaluation should be well established. While the simulation was 
based on specific learning outcomes derived from the literature on PT competencies, the evaluation methods were 
not as well established. Therefore, our second question, which was related to the necessary components of a PT 
simulation, cannot be addressed without additional data. While we have anecdotal data related to the fidelity of the 
simulation, and content validity from our expert review, the incompleteness of our data does not allow us to draw 
any empirical conclusions. 

While the outcomes from the study were mixed, we believe the study is important for the following reasons: 

• To provide alternative learning and assessment opportunities for non-traditional students. 

• To continue to look for strategies that increase critical thinking and problem-solving skills in our 
graduate students. 

• To add to the theory base of instructional strategies that promotes authentic learning contexts, 
specifically simulations. 

• To add to the growing body of empirical knowledge on developmental research. This is important for 
the continued growth of the field of Instructional Design. As Richey, et. al (2004, p. 1 102) states 
“Given this definition of the field, developmental research is critically important to the evolution of our 
theory base.” 
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Figure One: Course Design 




Figure Two: Simulation Model 
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Abstract 

What is the nature of onsite and online mentoring which enables student teachers to design 
inquiry-based, technology rich learning experiences? In this case study, faculty and expert teachers worked 
with fifteen student teachers during an elementary school practician. An online intelligent design 
environment supported the development of a community of practice and instructional design. Discussion 
focuses on teaching from an inquiry stance and engaging in the intellectual work of mentoring student 
teachers to teach with technology rather than just modeling practice. 

Context For Inquiry 

Preparing teachers today requires critical examination of what it means to teach and learn in 
increasingly networked, technology-rich classrooms. Most young people entering teacher preparation 
courses have not, themselves, experienced such schools. They and many of the students who will follow 
them for the next ten years, have been shaped by an education system that is still struggling to make the 
transition to teaching and learning in a post-industrial era. On campus and in schools, they are learning to 
be teachers guided by university faculty and experienced teachers who are, themselves, are only starting to 
come to grips with the pedagogical changes required to engage students in technology -rich learning 
environments. 

Living this transition, new teachers cannot depend either on their own experience of K- 12 
schooling, nor yet on the widespread and effective use of technology for teaching and learning in schools 
and in post-secondary programs to provide the images or the expertise they will need to move their own 
practice beyond currently conventional uses of technology. At the moment, technology use in schools and 
on campus tends to replicate and reinforce familiar practices of schooling: do an Internet search 
on. . .present your report in PowerPoint. . .word process your assignments. . .access your readings from the 
course outline. Even when they, themselves, are fluent technology users in their lives outside school, few 
student teachers express confidence about using technology in creative and interesting ways that transcend 
such practices (Clifford, Friesen & Lock, 2004). We have encountered few schools and university programs 
that devote critical and innovative attention to the ways in which teacher preparation courses and practicum 
experiences with ICT might help prepare student teachers differently for their future roles (Jacobsen, 
Clifford and Friesen, 2002). It is still too often the case that questions about technology revolve around 
matters of utilization rather than around questions of fundamental school reform. 

Learning with technology, as distinct from learning about technology, has the capacity to 
transform learning environments in ways that are difficult for most educators to imagine. Coupled with the 
struggle many adults have in using basic computer functions such as email, search engines, and 
presentation software is the much larger issue that the children in today’s schools have never known 
anything other than a digital world. For the first time in human history, the young are more confident and 
more fluent with the dominant technologies of the times than the adults charged to teach them. They are 
“digital natives’’ (Prensky, 2003) whose new abilities, skills, and preferences are to a large extent 
misunderstood and ignored by the previous generation of educatois who speak and act with a distinct 
digital immigrant accent. 

Finding ways to bring educators’ attention to the implications of digital technologies for learning, 
and to bring those technologies into classrooms in increasingly meaningful, effective and innovative ways 
is one of the important tasks of teacher education. As Cochran-Smith (2003) notes, “whether by design or 
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by default, then, this means that teacher educators — those who teach the teachers — are now the linchpins in 
educational reforms of all kinds (p. 5)” 

This paper is about a collaborative inquiry undertaken by an elementary school staff, a faculty 
member and a professional development organization to address the mentorship of student teachers by 
intentionally creating a culture of inquiry in the context of media and technology practice. This study is an 
example of design research that directly addresses the crucial intersection of teacher professional 
development, student teacher preparation and the practicum experience in both face-to-face and online 
environments (Bereiter, 2002). The questions we raise outline important research directions we have been 
following for a number of years. First, where technology is concerned, the old “follow the expert” model of 
teacher preparation becomes troublesome in significant ways. Currently, most experienced classroom 
teachers (that is, the ones who would conventionally provide models of exemplary teaching for practicum 
students) are, themselves, only beginning to learn how to think and work in new ways with technology. 
Furthermore, these teachers are caught in the transition to post-industrial practices of teaching and learning. 
Thus, taken-for-granted notions that field placements provide novices the opportunity to learn from the 
expert modeling of practice no longer hold. Because the expertise of experienced teachers is, itself, very 
much in flux, a very different approach to the practicum experience is called for, best captured in Cochran- 
Smith and Lytle’s (2001) term “inquiry as stance”: 

Learning from teaching through inquiry assumes that beginning and experienced teachers need to 
engage in similar intellectual work. Working together in communities, both new and more experienced 
teachers pose problems, identify discrepancies between theories and practices, challenge common routines, 
draw on the work of others for generative frameworks, and attempt to make visible much of that which is 
taken for granted about teaching and learning. From an inquiry stance, teachers search for significant 
questions as much as they engage in problem solving. They count on other teachers for alternative 
viewpoints on their work. In a very real sense, the usual connotation of "expertise" is inconsistent with an 
image of teacher as lifelong learner and inquirer. 

Second, exploring creative and innovative ways to use technology puts everyone into a place of 
genuine inquiry about school reform. Thus, professional development becomes an essential component of 
the work, which must exist along a continuum of professional scholarship for student teacher learning 
through to teacher in-service and graduate work. Teacher preparation experiences closely tied to the 
learning of experienced teachers provides a powerful environment for change, both for experienced and for 
beginning teachers. 

Third, field supervision of student teachers is not conventionally regarded as scholarly work. In 
many universities, field supervision is assigned to sessional instructors and seconded teachers, who are 
believed to be closer to “the reality of the classroom” and therefore particularly suited to model practical 
teaching expertise. Often, the major work of these instructors is to evaluate student teacher progress. In 
line with Cochrane-Smith and Lytle’s (2001) work, however, we argue that a scholarship for teaching 
demands the active involvement of faculty in intentional communities of inquiry where “everyone is a 
learner, a researcher, a seeker of new insights, and a poser of questions for which no one in the group 
already has the answers” (p. 23). 

Generally, practicum placements are arranged so that student teachers are assigned to individual 
teachers in schools with attention to their subject specialties, grade preferences and perhaps geographical 
proximity. In this project, a cadre of student teachers was assigned to a school along with a faculty advisor. 
The cadre placement was made (i) in order to create a diverse community of practice for the purposes of 
inquiring in a disciplined way into the changes to teaching and learning demanded by the effective use of 
ICT and (ii) to break down the conventional isolation of the classroom teacher in order to permit 
experienced teachers to work more effectively together; student and experienced teachers to function as 
teams; and the school to regard the cadre of students assigned to them as the responsibility of the school as 
a whole. 

Through interventions in the conventional structure of student teaching experience, it was hoped 
that the study would seriously address the complexity of change that the effective use of technology in 
schools both enables and requires, and that it would do so in a way that was better able to prepare new 
teachers for post-industrial classrooms. 



Research Method 

Using a case study research approach (Merriam, 1998; Stake, 1995) we documented the nature of 
mentoring relationships developed with 15 student teachers who completed an extended practicum in an 
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elementary school in Fall 2003. The faculty member used Intelligence Online (io), an online learning 
environment and instructional design tool, to cultivate professional dialogue about practice and to provide 
mentoring support in the design and development of inquiry -based projects and learner assessments. 

Riverbend Elementary School staff worked closely with a faculty member from the Faculty of 
Education to support a cadre of 15 student teachers in their extended practicum experience. Students spent 
4 full days a week in classrooms. The faculty advisor spent two full days a week in the school working with 
student teachers, with partner teachers, and with Galileo Educational Network professional developers who 
were working with teachers to develop strong inquiry -based practice. Intentional goals that framed how 
field supervision was defined by the faculty member were to facilitate a gradual and graduated, reflective 
and rewarding learning experience for student teachers, to observe student teachers in a variety of learning 
and teaching situations, to support classroom teachers and student teachers in establishing and sustaining 
mentoring relationships, and to facilitate mentor teacher assessments of student teacher growth and 
development. 

Community Building 

From day one, the student teachers and the faculty member were welcomed with open arms and encouraged 
to become active participants in the school community. Sixteen new individuals in a school can create some 
challenges, from parking to borrowing resources to finding a seat in the staff room! In August, the school 
principal and faculty member planned how to create a welcoming environment for the student teachers and 
designed ways in which staff would be supported in working with the cadre of 15 student teachers. The 
principal and faculty member assembled keys, staff handbooks, policy documents, media center loan cards, 
and other materials to support student teachers in becoming active members of the learning community. A 
classroom was made available for student teachers as a home base during the semester. 

Both the principal and the faculty member wrote letters of welcome to the student teachers that 
included information about the school, the cadre approach to field experience, the case seminar and initial 
expectations for classroom observations and relationship building. The faculty member also wrote a letter 
of introduction to each classroom teacher to thank them for working with student teachers, to explain when 
she would be in the school and to outline the schedule for case seminars in the school. The principal and 
faculty member hosted a welcome session very early in September for school staff and student teachers to 
discuss the cadre approach to the extended field experience, to provide an opportunity for people to meet 
and mingle, and to help student teachers to learn about the culture of the school. The pair also collaborated 
on the design of an early seminar with the school’s teacher librarian that helped student teachers to become 
aware of and make good use of the vast physical and technological resources in the school. 

Matching Mentees and Mentors 

Many of us can remember how important good mentors were to our early development. The need 
for mentoring new teachers is well documented in literature on teacher attrition (Gold, 1996). New teachers 
need continued mentoring and support in the field as they begin to experience and reflect on what it means 
to teach in technology-infused, inquiry -based learning environments (Jacobsen & Lock, 2004; Jacobsen, 
Clifford & Friesen, 2002; Jacobsen & Goldman, 2001). Our research has demonstrated that onsite and 
online approaches to professional development and mentoring that supports teachers in designing, 
implementing and evaluating technology rich learning environments for students can lead to transformed 
and sustainable classroom practices (Jacobsen, Clifford & Friesen, 2002; Friesen & Clifford, 2002; 
Jacobsen, 2003, 2002, 2001a, 2001b). 

Instead of randomly assigning student teachers to a mentor, school administrators and the faculty 
member worked together to support people in initiating and developing mentoring relationships amongst 
themselves. In the first week, student teachers were encouraged to introduce themselves to teachers and to 
observe and help out in a number of classrooms in their division. Mentor teachers welcomed student 
teachers into their classrooms to learn more about their instructional plans, inquiry projects, assessment and 
classroom management strategies and learning cultures. After the first week, the principal and faculty 
member began to formalize mentoring relationships and student teaching arrangements based on feedback 
from mentor and student teachers. While some found this approach to matching mentors and mentees 
stressful, widespread feedback indicated that most teachers and student teachers appreciated the extra time 
and opportunity to meet more of the children and teachers, to learn more about the school, and to establish 
mentoring relationships based on shared interests. 
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Continuous Professional Development 

The case seminar was unique in that weekly meetings took place each Tuesday afternoon at the 
school. The faculty member and student teachers had their our own classroom in the school which gave 
student teachers a place to gather for professional dialogue within and beyond case seminars, to collaborate 
on instructional planning and research and to access technology. Student teachers also congregated and 
collaborated with teachers in classrooms, in the staff room and in the media center before, during and after 
school. 

A meaningful and much valued outcome of holding the case seminar at the school was the regular 
participation by school administration, mentor teachers and a Galileo Network teacher. Dr. Sharon Friesen. 
Teachers and administrators facilitated several case seminar discussions about multiage classrooms, 
authentic assessment, class management and differentiated instruction. Sharon led a popular and valued 
seminar on issues to do with teaching elementary school mathematics. Teachers also hosted lunch hour and 
after school discussions about teaching issues and topics with the cadre of student teachers. The faculty 
member was able to serve as an academic resource and a liaison to the University of Calgary. School staff 
approached Michele with questions about graduate programs and continuous professional development 
opportunities on campus. Several teachers are pursuing graduate study in the Faculty, and others 
approached her with questions on preparing an application for graduate school. The faculty member was 
invited to lead a professional development session with Riverbend staff and student teachers on the 
relationship between teaching quality standards and narrative assessment of student teachers. 

IO - An Online Professional Development Environment 

The teachers at Riverbend School work in an online environment, IO (Intelligence Online) 
developed by Galileo Educational Network and Axia NetMedia (Jacobsen & Gladstone, 2004). 10 is a 
fully mentored online professional learning environment for student and experienced teachers. In addition 
to being fully mentored, IO is a situated learning environment (Herrington, 2002; Herrington & Oliver, 
1997; Herrington & Oliver, 2000; Herrington, Oliver & Reeves, 2002; Herrington, J., Oliver, R., 
Herrington, A., & Sparrow, H.,2000) in which teachers are guided through an inquiry -based instructional 
design process as they create authentic learning tasks for their students and develop performance 
assessments for these tasks. 

IO provides teachers with the tools and processes to choreograph a complex inquiry -based 
learning environment, and to communicate and learn with other IO members. 10 is designed to create and 
to support online professional learning communities that work. It contains 

• content to assist teachers in thinking about inquiry, 

• suggestions for how to involve students in meaningful ways right from the outset, 

• examples that illustrate each aspect of the design process, 

• a workspace for the developing inquiry task, 

• spaces for asynchronous communication with others, 

• a publishing feature that lets teachers create websites to involve parents and students in the 

evolving inquiry, 

• templates for the co-creation of assessment rubrics that map directly to tasks designed for students, 

• functionality to create class lists and student records, 

• mechanisms that permit online mentoring by Galileo Educational Network professional 

developers around the actual inquiries and problems of classroom implementation. 

Through IO, student teachers, faculty, cooperating teachers, and professional developers were bound 
together in ways that invited the student teachers into what it means to teach. IO provided a space where 
experienced teachers, professional developers, student teachers and faculty could all come together to 
design engaging work for the classroom and to work out the difficulties of meaningful practice together. 

When speaking about the value of having the support of 10, one of the student teachers stated, 
“...then with using IO ... being able to plan on that. That opened my eyes to what planning could be, 
assessment could be, [it] forces you to integrate technology in ways that I think I have even resisted in the 
past but also gives you the courage and confidence in yourself that you can because there are models to 
look at and show you how. And then you start to understand the benefits for the students. But you have to 
get past yourself as a stumbling block and understanding how it can be used” (Clifford, Friesen & Lock, 
2004). 
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During the student teachers extended field experience at Riverbend, 10 provided a place for 
asynchronous communication. University faculty, teachers, student teachers and experts in the field 
explored together the living character of what it means to teach from an inquiry stance. One of the 
preservice students reflected on the experience during a post-semester interview. “Yeah, it was very nice to 
go through that process with them and for it to be such an open discussion so that you are not sitting there 
thinking , ‘I'm not really understanding this. ’ It was recdly good to know that people who have been in the 
field for that long were having the same sort of difficulties and struggles” (Clifford, Friesen & Lock, 2004). 

Inquiry and Technology 

Researchers are only beginning to understand the value and potential of distributed learning 
environments made possible by networked technologies and are starting to document the range of uses to 
which these tools can be put to support and extend the student teaching experience in the field. Case 
seminars focused on Reflective Inquiry as a Sense-Making Process and living cases were based on key 
issues that arose from field experiences in the school. Student teachers were expected to regularly reflect on 
their teaching experiences, decisions and actions. In addition to maintaining daily reflections and 
observations in a field journal, weekly reflections on key questions, events and issues that arose during the 
week were submitted to the faculty member via email. In many cases, Michele responded to these weekly 
journals through email. 

Student teachers participated in discussion groups that the faculty member initiated in the 10 
online community that emended the case seminar discussion on several topics: classroom management, 
assessment, inquiry -based learning, and virtual education. The student teachers also took the initiative to 
host a range of online discussions in 10 about various topics within the cadre. 

Student teachers individually or collaboratively designed a technology enhanced inquiry project 
for children using Intelligence Online (10), a web-based instructional design environment. Although it was 
considered optimal for professional development that the student teacher had the opportunity to not only 
plan the inquiry project, but also to teach it, this was not a requirement given the variety of instructional 
schedules and plans that were in place in the school. Feedback on the inquiry projects was provided in four 
areas: instructional unit design, assessment rubric(s), academic rigour and active exploration. 

The Galileo Network supported the student teachers in the development of inquiry projects using 
Intelligence Online, in exploring innovative uses of technology for learning, and in finding rich resources 
and connections with experts to expand their instructional repertoires. Dr. Sharon Friesen contributed to 
onsite case seminars on a regular basis, co-planned instructional and assessment strategies with the faculty 
member, and supported and extended student teacher dialogue in online discussions. A rich community of 
inquiry developed from the sharing and professional development that evolved in the school between 
mentor teachers, student teachers, university faculty and Galileo Network professional development 
experts. 

Student teachers used a range of technologies in support of student learning across the curriculum, 
from digital cameras and the Smartboard, to Internet research, computer-based artwork and digital 
filmmaking. Several classes of division one students produced interpretations of Ted Harrison paintings 
using KidPix, a computer based art and animation tool. Student teachers learned how to support several 
division two classes that were involved in digital filmmaking projects. A number of student teachers 
researched, reviewed and assembled lists of web-based resources that supported student research and 
learning across the curriculum. Children across the grades wrote stories and poetry using the AlphaSmarts, 
which are mobile and inexpensive text editing devices that can connect to computers for more sophisticated 
editing and printing. 

Mutual Benefits of the Cadre Approach 

There are a number of benefits in the cadre approach for both school staff and student teachers. 
School staff appreciated having the field advisor in the school two days per week to discuss emergent 
issues and concerns, to celebrate student teacher strengths and accomplishments, to compare approaches to 
instructional planning and field journaling, to clarify university expectations and approaches to narrative 
assessment, and to ask about graduate programs. The regular onsite presence of the field advisor allowed 
for more communication and continuity between the mentor teacher, student teacher and the faculty 
member. Teachers recognized and appreciated the many ways in which the cadre of student teachers 
supported each other, shared information, plans, assessments and resources and introduced new ideas. 
Mentor teachers often commented on the benefits of articulating what they knew and held intuitively about 



312 




instructional planning, assessment and curriculum with those who were new to the profession. Teachers 
enjoyed sharing their expertise with the entire cadre of student teachers in a small group setting, such as 
case seminar or during a lunch hour session. 

Being part of a cadre offers a number of benefits to student teachers. The fifteen student teachers 
felt very connected to each other and appreciated the opportunity to build an authentic and meaningful 
mentoring relationship with their field advisor who was there two days per week. Student teaching, like 
other internships, involves emotional highs and lows and daily encounters with ill -structured (Jonassen, 
1997) and unanticipated problems. Becoming connected to an onsite and online community of peers who 
are going through a similar experience in the same place is reassuring. Student teachers drew on each 
others’ strengths while building relationships with children and teachers in the school and becoming 
acclimatized to the school culture. They appreciated the opportunity to work together as a group, to share 
curricular resources and expertise, and to critically discuss strategies and methods as they took on more 
teaching responsibilities. Student teachers valued frequent teacher involvement in case seminar and regular 
access to the school principal to discuss what was happening in the school, to analyze emergent issues and 
to explore living case topics. Student teachers posed questions, investigated decisions, and analyze possible 
responses and solutions as a learning community in case seminar, in online discussion groups and during 
thousands of serendipitous and planned meetings within and beyond the school. The student teachers 
recognized both the value of being a part of two different classrooms and observing two different teaching 
styles as well as the consistency of being with one partner teacher through the entire student teaching 
experience. This cadre of student teachers highly valued the opportunity to participate in the culture of 
inquiry that was establishing in case seminar and lived out daily by children and teachers. 

Student teachers benefited from their onsite and online collaboration with Galileo teachers, who 
supported student teachers with their instructional planning in IO by providing resources, teaching ideas 
and prompt feedback. Students valued Sharon’s participation in case seminars, and appreciated her helpful 
advice and expertise about teaching elementary mathematics. 

Developing a technology-enhanced inquiry project using IO increased student teacher’s 
understanding of inquiry, instructional planning and learner assessment. Working through IO challenged 
student teachers to think about the big picture and how to make projects meaningful for children. Student 
teachers became more aware of all the planning involved in doing an inquiry project by answering the 
questions and filling in the different sections in IO. Investing time in instructional planning using IO helped 
student teachers to intentionally create rubrics that connected learning objectives to the big ideas important 
to the subject or idea. The majority of student teachers agreed that other student teachers should have 
access to IO to learn more about instructional planning and assessment for inquiry -based learning. 

Discussion and Findings 

The nature of onsite and online mentoring that enables student teachers to use inquiry -based 
approaches to technology integration is responsive and flexible (Jacobsen, Clifford and Friesen, 2002). Our 
goal for using 10 was to connect student teachers with each other, with education faculty, and with 
professional colleagues who share an interest in questioning teaching practice, pursuing inquiry-based 
learning for children and themselves, and developing innovative uses of technology for learning. We 
established an online mentoring environment for student teachers with the goal of helping them to resist the 
urge to “teach the way they were taught” as they encountered the many challenges that come with learning 
through inquiry and technology. The online mentoring approach was flexible, responsive and emerged 
specifically in response to the needs and experiences of the student teachers themselves. 

Student teachers were required to develop one cross-curricular inquiry project for children that 
employed media and technology in creative and meaningful ways. Intelligence Online supported student 
teachers in their instructional design tasks of establishing curricular connections and goals, preparing 
essential questions and designing inquiry tasks, and developing performance and assessment rubrics. Each 
student created a technology -enhanced, inquiry project for children, either individually or in pairs. All of 
the student teachers reported that (i) they would recommend that other student teachers have access to the 
Intelligence online design tools to support instructional design and development, and (ii) that they would 
benefit from using this online design support tool as a beginning teacher. Student teachers reported that the 
online instructional design process took a significant amount of time but they expected that subsequent 
design tasks would take less time. 

We know from first hand experience with student teachers in 10 , that “design-based activities not 
only provide a rich context for learning, they also lend themselves to sustained inquiry and revision that 



313 




will help designers come away with the deep understanding needed to apply knowledge in the complex 
domains of real world practice” (Koehler, Mishra, Hershey, & Peruski, 2004, p. 32). Analysis and reporting 
of results focused on the efficacy of online design and discussion spaces for ill structured problem-solving 
(Jonassen, 1997). Key findings are that (i) Intelligence Online did support mentors and student teachers in 
sustaining meaningful professional dialogue throughout the semester, and (ii) the online design tools 
enabled student teachers to develop inquiry -based, technology enhanced projects for children. Student 
teachers participated frequently in instructor-designed, community discussion spaces that supported and 
sustained ongoing professional dialogue about a range of key pedagogical issues and concerns. In 
community and one-to-one discussions online, the faculty member and professional educators were able to 
provide intentional mentoring and sustainable support to student teachers that built and extended upon face- 
to-face encounters. Our analysis reveals that online spaces provide a risk- reduced space to work out some 
of the dilemmas and ill-structured problems that characterize early and ongoing teaching efforts. All of the 
inquiry projects designed by student teachers using Intelligence Online demonstrated acceptable uses of 
technology for learning. Further, most student teachers (75%) were able to develop robust and authentic 
inquiry projects for children that implemented innovative, creative and meaningful uses of media and 
technology. 
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Abstract 

The terms “cooperative” and “collaborative” are sometimes used interchangeably in reference to 
group learning activities in classrooms and in online settings. However , they can be viewed as differing in 
terms of characteristics such as pre-structure, task structure, and content structure (Strijbos & Martens, 
2001 ; Panitz, 1996). This study attempted to help clarify these differences and the effects of the two types of 
groups on learner performance in an online debate. The study investigated the effects of a highly structured 
cooperative learning (HSCP) group, which had pre-assigned debate positions as a pre-structure, 
argumentation scaffolding as a task structure, and evaluation scaffolding as a content structure, compared 
to a low structured collaborative learning (LSCL) group, which did not have these structures, in terms of 
pre-sen’ice teachers’ decision changes, critical thinking, and interaction patterns. Results demonstrated 
that there were greater amounts of critical thinking, and of critical and dynamic interaction patterns in the 
HSCP than LSCL group. 



Objectives or purposes 

The purpose of this study was to clarify the differences between cooperative and collaborative 
learning groups determine whether a highly structured cooperative learning (HSCP) group would have 
positive effects on individuals’ changes in decision making, use of critical thinking, and engagement in 
critical and dynamic interaction patterns compared to a low stmcture collaborative learning (LSCL) group. 
The groups were differentiated by high levels of pre -structure, task structure, and content structure in the 
cooperative group and low or zero levels of these features in the collaborative group. 

Perspectives or theoretical framework 

Cooperative and collaborative learning have often been used interchangeably without clear 
distinctions even in face-to-face settings. In online learning settings, most of literature regarding group- 
based learning has predominantly used the terminology of collaborative learning rather than cooperative 
learning, but has not provided empirical evidence of the groups’ structural characteristics or results. 
However, it is possible and perhaps useful to distinguish between the two types . Cooperative learning 
involves a group of people for a single task based on a structure with series of steps (Kagan, 1985), and 
mutual responsibility with highly structured and specialized roles (Kessler, 1992). On the other hand, 
collaborative learning is defined as a group activity where a group of people work together to create 
meaning, explore topics, or improve skills (Eastmond, 1995; Harasim et al., 1995) without forceful 
accountability of group learning which is unlike cooperative learning (Zvacek, 1991). According to Pantiz 
(1996), cooperative learning is highly structured, relates to more well-structured tasks for limited solutions, 
and requires the acquisition of a well-defined domain of knowledge and skills. Collaborative learning, on 
the other hand, is less structured, relates to ill-structured tasks for open and flexible solutions, and requires 
the acquisition of an ill-defined domain of knowledge and skills. Since there are varying ways of 
implementing these structural elements in the design of a group-based learning environment, this study 
operationalized two extreme group-based learning strategies in terms of three structural elements: pre- 
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structure, task structure, and content structure. The resulting two groups are high-structure cooperative 
(HSCP) and low-structure collaborative (LSCL), which is consistent withPanitz’s (1996) definition. 

Based on these structural differences, there were three ways in which the HSCP group was 
expected to perform differently from the LSCL group. The first pertains to changes in decisions made at the 
beginning of the debate compared to those made at the end. One of the main functions of group debate is to 
allow group members to compare their positions to those of others (Jellison & Arkin, 1977); and to 
evaluate or reevaluate alternative decision choices prior to group decision-making (Bernstein, 1982). 
Blumenfeld et al. (1996) also empirically reveal the significant effects of structure on online, group 
discussion in terms of achieving consensus and making better decisions compared with those online, group 
discussions characterized by unstructured discussion. Therefore, it was hypothesized that participants in 
the HSCP group would have more decision changes than in the LSCL group. 

Secondly, Paul (1993) suggests that the quality of critical thinking is determined by the quality of critical 
reasoning. Highly interactive and learner-centered debate based on roles and the provision of scaffoldings 
can affect both the process and outcomes of online debate (Cavalier et al., 1995; Hooper et al., 1993; 
Singhanayok & Hooper, 1998). Therefore, it was hypothesized that the HSCP group would have more 
improvement in critical thinking than the LSCL group. 

Thirdly, structured interactions are very effective ways to raise the level of student discourse and 
facilitate more student engagement (Smith, Johnson, and Johnson, 1981). Elaborative procedures and 
structures can also foster the benefits of debate in the group-based learning process (Smith, Johnson, & 
Johnson, 1981). The considerable assistance of social scaffoldings in the process of debate can guide 
students to consider alternative explanations, to negotiate complex issues, to evaluate progress, and to 
systematically offer justifications for their reasoning (Palincsar et al., 1993). Groups under structured 
interaction can be better supported in solving problems effectively with the help of explicit thinking 
processes with precise goals, planed procedures, generated alternatives, and repetitive modification process 
for better outcomes under the structured interaction process (Chang and Well, 1987). Therefore, it was 
hypothesized that the HSCP group would have more dynamic and interactive interaction pattern than the 
LSCL group. 



Methods, Techniques, or Modes of Inquiry 

One independent variable with two levels , high-structure cooperative (HSCP) and low-structure 
collaborative (LSCL), was implemented in an online debate assignment in terms of the amount of pre- 
imposed structure, task-imposed structure, and content-imposed structure. 

Online Group Debate with HSCP Design 

To create a high level of pre -structure , the researcher posted a proposition, which asserted that one 
of two WebQuests was more effective and efficient than the other for online debate. For the HSCP design, 
the researcher randomly assigned subjects to a pre -position of either ‘pro’ or ‘con’ with respect to the 
proposition for debate. For the low level of pre -structure approach, there was no pre-assigned position. 
Students, therefore, just chose and supported one of two given WebQuests, based on their personal 
preferences. To create a high level of task structure, argumentation scaffolding was given to provide 
structure to the task such that students were directed to insert appropriate labels for their comments 
whenever they posted a message during online debate. The instruction provided specific examples of 
propositions and possible types of message labels for debate. To create a high level of content structure, 
content specific scaffoldings for WebQuest evaluations were provided while subjects participated in online 
debate. Subjects performed the task of evaluating two different WebQuests, based on the given WebQuest 
evaluation criteria. Subjects evaluated the given WebQuests by referring to these descriptions of Web 
Quest evaluation specifications. 

Online Group Debate with LSCL Design 

For the online debate with the LSCL group, the researcher used the three subcomponents of low 
level pre -structure, ill-structured task, and open content structure, based on Panitz’s (1996) definition of 
collaborative learning. For the low level of pre -structuring, there were no pre -assigned roles or positions 
while subjects participated in the online debate. For the ill-structured task, no argumentation scaffolding 
was given. For the content structure, content specific scaffolding for the evaluation criteria was given to 
the students. 
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Data sources or evidence 

Participants consisted of forty -four students enrolled in an undergraduate pre -teaching course in 
educational technology at a large southeastern university in the USA. Four of the subjects were eliminated 
from data analysis because they did not participate in one or more of the pre-test, debate, or post-test 
activities. Decision-making, critical-thinking, and interaction patterns were measured in order to determine 
the effects of HSCP and LSCL design on online debate. 

In order to measure decision changes, the evaluation scores in both the pre-test and post-test were 
compared and the difference between HSCP group and the LSCL group was examined using a t-test. The 
change in decisions was measured by three elements of the pre-test and post-test as described in the results. 
The increase in critical thinking was measured by comparing the rationales written in the pre-test with 
those in the post-test, regarding the strengths and the weaknesses of each given WebQuest using the same 
WebQuest Evaluation Worksheet. The significance of the differences in the improvement of critical 
thinking was investigated using an independent sample T-test. In order to have consistency in grading the 
comments, the two graders engaged in considerable discussion. A Spearman Correlation- Coefficient test 
was conducted to establish inter-rater reliability between two coders. 

For the investigation of online interaction patterns, the critical event sequences were observed. 

The researcher calculated the frequency of message event sequences based on the coding that was 
developed to correspond with Toulmin’s model of argumentation (Toulmin, 1958) with the following 
matches (see Figure 1): “claims” corresponds to “proposition,” “warrants” corresponds to “argument,” 
“backing” corresponds to ’’evidence,” “rebuttal” corresponds to “critique,” and “qualified’ corresponds to 
“explanation.” Another category, called “Others,” which does not occupy a position within the diagram, 
indicates any type of message that does not fall into any of the categories. It was not necessary to use the 
“Others” category in the present study. 
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Data 
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Critique 

(Rebuttal) 



Figure 1. Debate Coding Plan based on Toulmin’s (1958) Model 

In the process of online debate the critical event sequences were observed in order to examine the 
critical reasoning process. The researcher observed the frequency of the following event sequences: 
argument-argument, argument-evidence, argument-critique, critique-critique, critique-evidence, critique- 
argument, evidence-argument, and evidence-critique based on the literature regarding the process of critical 
thinking (Paul, 1993; Derry et ah, 2000). The researcher applied an independent sample Chi-Square test to 
determine the significance of differences in the critical and dynamic event sequences and also calculated 
Cohen’s Kappa (1960) to establish the inter-rater reliability between two coders. 

Results and/or conclusions/point of view 

The three dependent variables examined in this study were: 1) decision changes in determining the 
better-designed of two WebQuests, from initial decision to decision after online debate; 2) the level of 
improvement in both the quantity and quality of critical thinking (the differences in critiques and evaluative 
comments between pre-test to post-test; and 3) the interaction pattern based on event sequence analysis for 
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critical interaction processes of the online debate messages stored in the online discussion board for three 
weeks. 

Contrary to the hypothesis, the total mean score for decision change was 5.16 (SD = 3.14), from a 
possible range of 0-to-15. The descriptive statistics revealed that the mean score for decision change in the 
LSCL group (M = 5.41, SD = 3.84) was slightly higher than in the HSCP group (M = 4.91, SD = 2.31). 
However, the result of an independent sample T -test based on all three elements of decision change 
revealed no significant difference between the two group means (t (42) = .524, g. > .05). At the same time, 
there were no statistical significance in the results of any of the three elements as follows: the first element 
of decision change based on the perceived quality of the first pair of WebQuests (t (42) = .000, g >.05); that 
of the second element of decision change based upon a comparison of the decisions regarding the better- 
designed WebQuest (t_(42) = -.592, g >.05); and that of the third element of decision change regarding the 
reflected change of rank order of the first two, out of six, WebQuests (L(42) = .815, g > .05). 

The total mean score for the development of critical thinking was M = 1 1.52 (SD = 9.36). The 
improvement of critical thinking was much higher in the HSCP group (M = 14.05, SD = 9.59) than in the 
LSCL group (M = 8.46, SD = 8.43). This finding did support the hypothesis (t (40) = -2.21, g < .05), at the 
alpha set at .05 with the power .58 for determining large effect size in a two -tailed test. The inter-rater 
reliability between two coders in grading the improvement scores of critical thinking, the Pearson 
correlation-coefficient test was conducted with alpha set at .01 with the significance of ? (44) = .000, g < 
.01. Therefore, there was high reliability between two graders in measuring the critical thinking skills 
using the WebQuest Evaluation Worksheet. The proportion of variance of coefficient value was .98 (? 2 = 
(•99) 2 ). 

The total mean score for the observed frequency of critical event sequences for both groups per 
week, including all the possible types of event sequences between argument, critique, and evidence, was 
104.67 per week. The mean score under the HSCP condition was higher (M =61) than that of the LSCL 
condition (M = 43.67). This finding was consistent with the hypothesis based on a Chi-square test for two 
independent samples (? 2 (6, N= 44) = 18.479, g < .05) with g value = .005. Based on the Cohen-Kappa test 
for inter-rater reliability between two coder’s coding in LSCL group, the strength of agreement was 
considered “very good” with a Kappa value of 0.840. For the HSCP group, the inter-reliability between 
first coder’s coding and the subjects ’ coding was found to be “Good,’’ with a Kappa value of 0.735. The 
confidence interval was 95% in both tests (Cohen, 1960). 

Educational or scientific importance of the study 

This study concludes that the HSCP learning strategy as distinguished fromthe LSCL strategy 
based on pre -structure, task structure, and content structure is statistically significant in facilitating group 
members’ critical thinking as well as dynamic and critical interaction processes in an online learning 
environment. Several significant issues are identified in this study that may contribute to a more reasoned 
approach to the design of strategies for online, group-based learning as follows: 1) a more rationalized 
approach to the terminologies involved in collaborative and cooperative learning, 2) a clearer identification 
of the characteristics of strategies employed in collaborative and cooperative learning, and 3) an analysis of 
online, group interaction patterns in order to establish the key elements involved more concretely. The 
unique characteristics of three sub -components of cooperative learning employed in this study (pre- 
structure, task structure, and content structure) can be effectively implemented in designing group-based 
learning for online learning environments especially for the increase of critical thinking, and more dynamic 
and interactive online group interactions. 
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Abstract 

Today, undergraduate students enter college with increasingly more sophisticated computer skills 
compared to their counterparts of 20 years ago. However, instructors are still using traditional 
instructional strategies to teach this new generation. This research study examines strategies that 
instructors employ to teach introductory computer literacy classes in higher education. We explore 
alternative teaching methodologies in an effort to close the gap between classroom practice and real-world 
application. 



Introduction 

Over the past twenty years, essentially all undergraduate education programs have replaced their 
state -mandated audio-visual course with a required introductory computer course. During the 1990’s, most 
college students had little or no experience using computer software, but today, most incoming college 
students have been exposed to an introductory computer literacy course in middle school or high school. As 
a result, college freshmen enter undergraduate technology courses with increasingly sophisticated 
technology skills. Instructors now face an increasingly difficult pedagogical challenge to accommodate 
both novice and advanced students in the same introductory computer course. Some students can barely 
insert a diskette while others are fluent in computer programming. Teaching to the “middle” is 
unsatisfactory: beginners struggle, and become overwhelmed while advanced students become bored, and 
unchallenged. This paper describes experimental instructional strategies to improve pedagogy and 
accommodate heterogeneous student populations in introductory computer courses. 

We will begin by describing how computer literacy has evolved, then we will review instructional 
strategies employed in computer literacy courses to accommodate heterogeneous learners, next we will 
describe the existing course, and lastly, we will discuss the course redesign and initial results. 

Computer Literacy 

The definition of computer literacy has evolved over the past twenty years. In the 20 th century, 
computer literacy courses focused on acquiring conceptual knowledge about how computers worked and 
gaining technical skills in using standard computer applications, such as word processing, electronic 
spreadsheets, and e-mail (Computer Science and Telecommunications Board, 1999). Today students must 
grasp the underlying principles of the technologies and understand how those principles relate to real-world 
tasks. Learners must be able to integrate their knowledge of technology skills, analytical skills, and critical 
thinking skills to solve complex problems. Yet, the emphasis of many introductory computer courses 
continues to focus on ensuring that students can effectively use a word processing program to prepare 
academic papers in a specified format. We believe that teaching skills which students have already 
mastered is not a good use of limited instructional time. 

Outdated Instructional Methods 

The traditional demonstration approach employed to teach introductory computer courses has 
remained essentially unchanged for the past twenty years. The instructor demonstrates how to perform a 
task step-by-step followed by the students practicing the behavior. Typically, once the students acquired 
basic proficiency in using one software application, the instructor would move on to the next software 
package. In a relatively short period of time, students were exposed to a broad range of software 
applications, but none of these software programs was covered in much depth. Although the traditional 
demonstration approach is efficient, students remain passive and dependent on the instructor for guidance 



321 




(Hadley, 2002). In addition, half the students’ report that the instructor’s pacing of software demonstrations 
is too fast, while other students complain that the pacing is too slow — novices become frustrated, unable to 
keep up while advanced students become impatient and lethargic. More troubling, the demonstration 
approach is incongruent with current workplace demands in which employees are expected to acquire new 
technology skills independently through self-study (Goldsborough, 2003). According to Hadley (2002), the 
demonstration approach also severely limits opportunities for students to collaborate or participate in active 
learning strategies. 

Instructional strategies such as pairing advanced students with novice learners, allowing advanced 
students to proceed through the course material independently, or providing advanced students 
supplementary software is a “stop-gap” compromise at best. At the same time, the advantage of using a 
traditional “one size fits all” approach is that all students exit the introductory computer course with a 
consistent foundation and a common technical skill baseline. 

Strategies Employed to Bridge the Gap 

Eventually, the skill gap of incoming freshman will narrow as state-mandated changes to K-12 
computer literacy curriculums are implemented throughout the US (U.S. Department of Education, 2002). 

In the interim, however, colleges and universities have employed a wide range of strategies to 
accommodate their heterogeneous student populations. Some institutions allow incoming freshman to 
waive taking a required computer literacy course by successfully passing a skills assessment test. Other 
institutions provide quick-start workshops rather than requiring students to take a computer literacy course. 
While a “test out” option is useful for assessing technical skills, for those instructors integrating the 
application of technology concepts into performing real-world tasks, an easy-to-score test is impractical. 

The quality of commercial technical skills assessments also varies widely. For example, expensive 
computer-based systems simulate students’ performing authentic tasks while less expensive assessments 
use multiple choice questions that ask the individual to recall inert knowledge (e.g., ask which pull-down 
menu to use to select a particular command). These types of assessments do not adequately test whether the 
students are capable of performing the application software skills. 

Our research also found one teacher education program that divided their computer literacy 
requirements into three separate 1-credit courses. The first course (which students have the option of testing 
out of) focuses on acquisition of technical skills. The second 1-credit course focuses on the application of 
skills, and the third component involves the integration of computer and media skills within the teaching 
methods courses. The students acquire computer literacy skills within a broader context that focuses on 
direct application to the student’s major course of study. 

The Existing Course 

This research is based on data gathered from a required introductory computer course taught at 
two research universities and one comprehensive college. The course is taught face-to-face and online for 
education and non-education undergraduate majors. Students usually take the course during their freshman 
year (or the first semester they transfer into the program). The course is taught in a computer equipped 
classroom with one student per computer and each course section consisted of 19-24 students. 

The purpose of the course is to introduce students to professional tasks, not simply to master 
technical skills using computer software. For example, students learn to create a school newsletter or 
employee biography incorporating technical elements such as footers, multiple columns, and graphics. 
Software applications include Microsoft Office (Word, PowerPoint, Excel, and Access) and Internet 
Explorer. In the comprehensive college’s version of the course, students are also introduced to workplace 
tasks involving Microsoft Visio, Microsoft Project, and Adobe InDesign; advanced students can also 
explore Adobe Illustrator and Adobe PageMaker. Technology concepts (e.g. computer hardware), ethics 
(e.g., copyright), and social concerns (e.g., digital divide) are also covered in all versions of the course. The 
required textbook includes clear step-by-step instructions and is accompanied by relevant workplace- 
oriented practice exercises designed to appeal to 18-24 years olds. The textbook also prepares the student 
for the Microsoft Office Specialist Certification exam. 

Course Redesign 

Based upon our observations and the data we collected, we determined a need to close the gap 
between the methods we traditionally employ to teach computer technology course and the natural 
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approach that students appear to use to become computer literate. We have begun to rethink how we teach 
the course and how we encourage students to take control of their learning. Using the introductory 
computer course at the comprehensive college as our test site, we began to redesign the course by making 
three key changes: ( 1) a hands-on computer skills assessment test was administered at the beginning of the 
course, (2) a flexible class attendance policy was implemented, and (3) the required textbook was 
eliminated. During the second week of the semester, each student met with the instructor to review the 
results of the computer assessment and to develop an individualized learning plan. Shortly after midterm, a 
second meeting was held with each student to review progress and make any necessary adjustments. 
Approximately one-third of the class sessions were mandatory to ensure students attended to conceptual 
content and to introduce new, unfamiliar software programs. All remaining class sessions were optional. 

We expected that small groups of students would attend the software demonstrations and that students 
would make greater use of learning resources such as print guides, videotapes, online tutorials, and 
coaching from a graduate assistant. We also expected that advanced students would work independently. 
We eliminated the required textbook because we found that once the technical skills were mastered, the 
textbook had little, if any, future value for the students. 

Initial Results 

As expected, results from the computer assessment test indicated that most students entered the 
course with varied skills in using Microsoft Word and PowerPoint. About 30% had some experience with 
Excel and less than 10% had experience with Microsoft Access. A small number of students (less than 
10%) were deficient in basic word processing skills. During the assessment test, we observed many 
students attempting to figure out how to perform various tasks by using Microsoft’s help feature. More 
interesting, the assessment test caused most students to reassess their computer competence more 
realistically. Several students commented that they thought they knew more than I did. 

Instructional Strategies Employed 

Flexible Attendance Strategy 

Traditionally, attendance was mandatory since seating capacity in computer-equipped classrooms 
was limited and significant class time involved hands-on practice. Attendance was also mandated because 
we believed that students who did not attend class regularly performed poorly or failed. Yet, while we 
required class attendance, we also espoused that the student take control of his or her learning — our policy 
was incongruent with our belief. Surprisingly, when we eliminated the attendance policy, most students 
(including advanced students) consistently attended class. Students like being given control of and 
responsibility for their attendance. Students cannot become self-directed learners when we mandate 
attendance. 

Use of Alternative Instructional Resources 

Over the past several years we used a high-quality, full-color text from a major college publisher, 
complete with practice exercises directed toward college students. We abandoned all textbooks because 
once the skills were acquired the textbook was no longer needed. We also noticed that our bookcases were 
filled with obsolete computer texts and manuals that we rarely, if ever, used for reference. Instead, we 
provide a broader variety of learning resources such as audiotapes, videotapes, print-based tutorials with 
CD-ROMs, web-based tutorials, and hands-on workshops. We also encouraged students to use the built-in 
help features of the software, as well as human resources (e.g., classmates, friends, graduate assistants, tech 
support, and even parents) — mirroring how employees learn on-the-job. At present, however, we are 
somewhat surprised that students seem to prefer to attend traditional software demonstrations rather than 
engage in self-directed study. 

Use of Learning Teams 

Another strategy we introduced fall 2004 was the use of student learning teams. Our intent was to 
promote self-directed learning and to cultivate a learning community. Each team was responsible for 
delivering a workshop on a software program such as Microsoft Visio, Adobe Illustrator, or Microsoft 
Project. A job aid was provided to each team that explained how to prepare the workshop. Suggestions on 
the job aid included beginning the lesson with an overview of what the software is used for, showing 
examples of documents created with the software, and creating hands-on exercises based on real-world job 
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tasks. To learn the software, each student team was provided print-based tutorials and CD-ROMs (e.g., 
Microsoft Step-by-Step). The teams were also provided a short list of specific technical skills that they 
were expected to cover in their workshop. Additionally, they were required to meet with the instructor 1-2 
weeks in advance of their workshop to review the context they had created for the lesson. The instructor 
also used the team meeting to reduce the potential for presentation anxiety and other difficulties. The 
instructor shifted his role to become a coach, and an instructional consultant rather than merely a resource 
for answering technical questions about the software. So far the results have been mixed. Students’ limited 
experience presenting a lesson reduced effectiveness and efficiency. 

Discussion and Conclusions 

As computer literacy becomes mandatory in K-12 curriculums, the need for required introductory 
courses in higher education could become unnecessary. The current generation of students entering higher 
education incorporates technology naturally and transparently into their daily life with little difficulty while 
instructors are struggling to make sense of rapidly changing computer technologies. Instructors must not 
only integrate technology into their teaching; instructors must rethink how they teach. As a result, today’s 
digital divide is less concerned with economic conditions and access to technology and more concerned 
with an individual’s technical and conceptual competence. A new digital divide has emerged out of the 
students’ abilities to naturally adapt new technological advancements and the instructors’ inability to adjust 
their instructional modes and strategies to meet the ever-changing capabilities of increasingly sophisticated 
and technology-savvy students. 
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Introduction 

Perhaps this paper is better titled, "Metadata; or, bringing them back to the card catalog," because 
that is exactly what we want to do. 

Think about the old days, the days before the technological advances of computer systems. Think 
about the hours of working with students to help them find the information they wanted by looking through 
the card catalog. Think about the extreme satisfaction on the part of both student and librarian as the 
desired book is located by using good library skills. 

Things are different today. Although teaching students how to use the catalog is no less important 
now than it was in the pre -computer days, computer technology affects not only how our students get to the 
books in our libraries but also how they get to electronic information. Many of us have electronic databases 
that now comprise the first line of attack for research questions. However, too often the second (sometimes 
even the first) line of attack is "THE WEB." Internet access to information has lured students away from 
beginning first at the catalog despite our best efforts. As long as the catalog remains a secondary resource, 
no matter how many books are required for an assignment, students will not look at it as a gateway to 
information. We must make the catalog useful again; we must make it as alluring as the Internet. We must 
make it so that the OPAC (Online Public Access Catalog) itself takes the students beyond the library walls 
so that they no longer need to look first in the OPAC, then to the bookmarks, then to the databases, and 
then to the Internet. We must make the catalog the place for one stop information shopping. To do that, we 
must include in the OPAC records for electronic information packages. 

The purpose of this paper is two -fold. First, is to familiarize the reader with some advances in 
cataloging that may not be so esoteric in the near future for school librarians. Second, is to provide hints for 
easy cataloging of traditionally uncataloged electronic information and thus enhance information access 
through the school library OPAC. 

Information Packages and Metadata 

In the beginning, we cataloged books, and only books. Then we started to include other items in 
our catalogs such as filmstrips and sound recordings. Catalogers today us the phrase “information package” 
to refer to anything being cataloged be it a book, a puppet, or even part of a web page. It is simply easier to 
say that we catalog information packages than it is to say we catalog all of these separate things. In fact, the 
phrase information package goes well with attempts to get catalogers away from thinking about cataloging 
types of items and into thinking about simply organizing information. If we think only about information 
organization, perhaps it will make the prospect of cataloging web sites a little less scary. 

If organizing information is in fact our goal, then we should be interested in organizing all types of 
information. An informal survey (Kaplan 2004) of members of the American Association of School 
Librarians revealed that, although librarians thought that electronic information was an important part of 
the research process and that metadata, if not now, would soon be an important issue for school libraries. 
Despite these facts, on the whole, electronic information (web sites) was generally not included in their 
catalogs. There were two major factors for not cataloging web sites: l.That web site URLs change all the 
time; and 2. There wasn't enough time to catalog web sites. With respect to changing URLs, I am reminded 
of the arguments given for not cataloging paperback books because they don't stay on the shelf very long. 
This may have been a viable argument in the days of hand cataloging but as automation took over and 
cataloging became a speedier process more and more librarians happily cataloged their paperback books. 
Time is always a factor in everything we do but it seems to be especially important in cataloging. With 
almost anything else to do, school librarians simply do not have the time to catalog information packages, 
especially those items that seem to be difficult to catalog. Not so long ago, those difficult materials were 
videotapes. No one wanted to catalog them because they were too hard to catalog and there was no time to 
do it. As we were once reluctant to catalog videotapes we are now reluctant to catalog web sites. As we 
once found that suddenly we could no longer remember all of the videotapes in our collections, we will 
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soon see that we will be unable to remember all of the sites for which we have created path finders or 
bookmarks. It is impossible to catalog all web sites but then we don't need to catalog all web sites. What we 
need to do is to have control of a few key sites that will help our students retrieve the information they 
need. As we select the books we have on our shelves, so too can we select the web sites we have in our 
catalogs. But first we must understand the nature of this electronic beast. 

Metadata is, literally, data about data. In some libraries, we have been cataloging metadata for 
years. Creating a record for a part of a book, rather than the entire book itself, that is cataloging metadata. 
The Reader's Guide to Periodical Literature is an example of metadata cataloging. The way we retrieve 
clip art for our PowerPoint presentations is also a result of metadata cataloging. In today's cataloging 
world, metadata is usually used to refer to electronically based information most specifically that which is 
found on the Internet but it can be any digital information (e.g. clip art in a file). It is helpful to catalog, or 
in some way attempt to control, this information precisely because it is so seemingly uncontrollable. With 
declarations that AACR (Anglo-American Cataloguing Rules) and MARC (MAchine Readable Cataloging) 
are too oriented to book cataloging and not flexible enough to deal with the idiosyncrasies of electronic 
information packages (Tennant, 2004), information managers have worked hard to develop more flexible 
means of organizing electronic information. Standards, such as SGML (Standard Generalized Markkup 
Language), XML (e-Xtensible Markup Language), and TEI DTD (Text Encoding Initiative Document 
Type Definitions), have been developed to create a way for catalogers to transfer data about the information 
package into a syntax understood by computer programs for information retrieval purposes. Actually, 
MARC is an example of such a communication tool however archaic it may appear to be to cataloging 
theoreticians today. 

It is exciting to see the changes in cataloging theory however; the practice of cataloging in school 
libraries is really dictated more by the behavior of the automated library systems than it is by theoretical 
advances. Therefore, in school libraries, at least for now, we are left to using the MARC format for 
information retrieval and AACR for information organization. For most of the general school purposes, I 
do not think that using MARC and AACR is such a bad idea. While it may be that in the not so distant 
future, our school library automation programs will accommodate more complicated cataloging formats, 
for the time being, it appears that the rumors of the demise of MARC are greatly exaggerated. 

OPAC vs. Internet 

The question you may be asking is, "Why mess at all with metadata? Why not depend on search 
engines, integrated databases, and such for access to electronic information packages?" The answer simply 
is that the catalog was developed on the basic premise of matching the user to the needed information. The 
Internet was not built on that premise. Also, we want students to know the whole array of information 
available to them, not just the web sites. But, students do not want to look first in the OPAC for the books 
and then on the Internet for electronic information. And who can blame them? We cannot possibly hope to 
catalog all Internet information, or even a fraction of it. But then, do students really need to get all of the 
information? Have they ever had access to all information? No! Our size and budgets have made it 
necessary to pick and choose what we have on our shelves. Similarly, our time, the reliability of the site, 
and the focus of our curriculum will dictate what Internet information we do and do not add to our catalogs. 
The point is that if the information is on the catalog then the students need to search only one place and will 
then spend a lot less time aimlessly surfing the Internet for information. Additionally, by cataloging web 
resources, we apply our techniques of collection development and avoid erroneous web sites that will lead 
students astray or, at best, give them poor information. Allow me to focus on one site as an example to 
illustrate my point. 

Case in point 

Mrs. Brown’s eighth grade science class is studying the human body. In small groups, students are 
to find print and non-print information about various parts of the body. Group A has to find out about the 
skull. One question is how many bones there are in the skull. The students jump onto the computers and 
search using the phrase, “bones in skull,” and the engines Google, Dogpile, and AskJeeves. One student 
looks on an online encyclopedia and one student looks for books on the school OPAC. The student who 
searches Google (http://www.google.com ). comes up with over 400,000 sites. After reading through three 
pages of results, she finally comes up with two potential sites with information: one is a free encyclopedia 
( http://encvclopedia.thefreedictionarv.com/skull ) that has some good information and a black and white 
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drawing the students might be able to use. While she’s there, she sees if she can link to the Classmates.com 
site that is advertised at the top of the page. The other site she finds a not very useful definition for bones 
from a Webster’s online site ( http://www.webster-dictionarv.org/definition/bonesl . Nearly every other site 
listed is about some organization known as Skull and Bones and so, by the third page, she’s had enough. 
The second student searched using Dogpile ( http://www.dogpile.com ). This student is buoyed by the fact 
that only 75 sites were retrieved, a much better hit than the 400,000 from Google. Plus, Dogpile offers a 
suggestion box and the student clicks on “What are the bones for the human skull?” This “refinement” 
yields two additional sites (77) but at least there are very few on the first page having to do with that Skull 
and Bone society. Dogpile is a more successful search with more relevant sites right on the first page. One 
site in particular. Skull Anatomy Tutorial from GateWay Community College in Phoenix, Arizona 
( http://www.gwc.maricopa.edu/class/bio201/skull/skulltt.htm) seemed to have the best and most easily 
understood information and it was second on the list so it was found fairly quickly. The student looking on 
AskJeeves, ended up with over 300,000 sites with the sponsored commercial sites listed first. He too made 
use of the refinement feature, “Bone Human Skull” and thus narrowed his search to 127,000 sites. Skipping 
the e-bay and Skull and Bones sites, he too ran into the skull tutorial as well as a very cool looking site 
from KidzWorld ( http://www.kidzworld.com/site/p922.htm ). Now the students have a problem because one 
site says there are 22 bones in the skull and another says there are 30. A search on World Book Online 
brought up 49 references that were so diverse, the students changed the search to just “skull.” This time 
their search was much more productive. They found an article that confirmed there are 22, not 30, bones in 
the skull and they have a nice graphic to go with this finding. Finally, they searched the OP AC and found 
one book on the skeletal system. By now the students had really had enough. Their Internet searching was 
fun but in all the time they spent looking at the wrong sites, they found two useful sites but with conflicting 
information. Additionally, they has to sift through a lot of commercials to get to the helpful sites and one 
student thinks she is now signed up to get email for Classmates.com. They completely forgot that the 
librarian had a pathfinder for Mrs. Brown’s science class and so they didn’t get to the sites that way. 

What could be done to help these students? I suggest that cataloging the useful sites instead of 
putting them on yet another place for students to go to is a good solution to this surfing problem. If these 
students only had to go to the OPAC to get to book and electronic information instead of searching in all of 
these different places, they might have more energy to complete their assignment. So now the librarian has 
to deal with metadata. However, in a way, he or she has already done that by creating pathfinders or 
bookmarks. Some bookmarks are more elaborate than others. Peter Milbury of Chico (California) High 
School has an extensive web site of helpful sites for his students ( http://dewev.chs.chico.kl2.ca.us/ ) (see 
figure 1). He has made use of HTML to organize his sites. Students in his school do not have to surf the 
Internet to find useful information, yet, they still have to search for books in a separate step using the 
school’s OPAC. Since XML and HTML are ways of dealing with metadata, the website set up by the 
librarian has already set him up for the next step, that of taking the web sites off of the pathfinder or 
bookmark and into the OPAC . 



Cataloging hints 

Cataloging not just the parent site but also sites within the parent site is not a difficult matter. Most 
automated programs have a template option for electronic resources. What we need to do is focus on the 
positive aspects of making these resources available to our students from a single source (the catalog) 
rather than from multiple sources; bookmarks, subject pathfinders, or search engines. Yes we are dealing 
with some animal that the academics call metadata, but to our students we are just making a site accessible 
through the OPAC. 

A few caveats: Be reasonable about the sites you select to catalog just as you are reasonable about 
those you include in your bookmarks. Government and education sites tend to have more stable addresses 
than do commercial sites. If changing URLs is a concern, stick to sites you are fairly certain will maintain 
current sites and stable locations. Most library automation programs allow the cataloger to select the type of 
information package being described. This is an important feature and should not be ignored in any kind of 
cataloging. Before beginning the cataloging process, be sure the record being created is for electronic 
resources. In cataloging electronic information packages, Taylor (2004) states that it is not reasonable to 
refer to a bibliographic record, the problem stemming from the root of the word, biblio -, meaning “book.” 
She suggests instead the use of the phrase, “surrogate record” to refer to the record that stands in the place 
of the item itself. Here the item can be anything, a book, film, or web site. 
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I have often been asked how to catalog a videotape that contains two separate programs. Happily, 
in cataloging metadata that is no longer a problem. We can catalog the main NASA page as easily as we 
can catalog one of the links with in it. Because we don’t have to worry about where the item will be located 
on the shelf or whether there will be duplicate call numbers, we are freer to catalog with more detail. Let us 
use the NASA site as a template for cataloging metadata using the MARC communications format. 

Like a book, our surrogate record begins with the 245 tag. Consider the home page for NASA (see 
figure 2). Notice the prominence of the words, “National Aeronautics and Space Administration” with the 
NASA logo adjacent. We will take that as our main entry. Like most films, most web sites will not have 
personal or corporate names for main entries. We might want to consider the NASA logo as other title 
information. To include National Aeronautics and Space Administration as a statement of responsibility is 
redundant. Thus our 245 will look like this: 

245 00 laNational Aeronautics and Space Administration Ihfelectronic resource] 

:lbNASA. 

We will also want an additional title entry for NASA: 

246 30 laNASA 

Moving down the MARC record, we need to describe the publication data. A small variation from 
book cataloging is the reference to date of publication. If it is clear (or known) when the site came online, 
include that information in the Ic of the 260 tag. If not, the Ic is left blank. 

260 _ laWashington, D.C. :lbNASA. 

Since the information package does not exist physically anywhere, there is no 300 tag. In some 
school library automation programs, this may be a problem. Some program demand information in the 300 
tag or the surrogate record cannot be saved. If this is the case, one might just add a descriptor such as “1 
web site” in the la to take care of the problem, even though it is not technically correct. Sometimes, due to 
the inflexible nature of most school library automation systems, we have to be a little flexible ourselves. So 
if possible, we will skip the 300 tag and go on to the 5XX tags. The first tag to enter is the 538 for access 
information. Because we are focused only on Internet sites, our entry is simple: 

538 laMode of access: Internet. 

The second 5XX tag is a 500 for source of title. Included here is the screen used for deciding on a 
title and when that screen was accessed. We are picking out the most stable site we can but even stable sites 
change their look now and then. Therefore it is important to inform the user when the site was accessed just 
in case we may need to account for changes in site information. We also include for the user’s reading 
pleasure, the date the site was last updated. This can be very important information to the user. One wants 
to make sure the site is viable and wasn’t just put up and forgotten. In our example site, we see that NASA 
is very good at keeping the site current. 

500 laTitle fromhome page (viewed on Oct. 13, 2004; last updated Oct. 13, 2004). 

In cataloging web sites, a summary statement is particularly important, especially if the site 
contains a variety of information. Many educational and government sites have descriptions of the 
objectives and goals of the site. These statements, often found in the “about us” links, can be copied and 
pasted into the 520 tag. One should look for keywords that will aid in the retrieval of the surrogate record. 
One might also include a 505 contents note that maps out the parts of the site. 

520 laOfficial site of NASA, with extensive links to NASA projects and information 

about the United States space program. 

505 0_ laLife on Earth - Humans in space - Exploring the universe. 

The subject area is where we really see the benefits of cataloging parts of the site separate from 
each other. Imagine the number of subject headings one would need to cover even half of the content of all 
of the sites put together. But, if the parts are cataloged separately, then a few broad headings are sufficient 
in describing the intellectual content of the parent page. 

610 10 laUnited States. IbNational Aeronautics and Space Administration. 

650 _0 laSpace flight IxHistory. 

650 _0 laSpace shuttles IzUnited States IxHistory. 

The final part of the surrogate record is the 856 tag. Electronic Location and Access. As the title 
describes, this is where one enters the URL for the site. The first indicator will be 4; meaning that the 
information package being described is accessed through http. The second indicator is the relationship of 
the URL to the item as a whole. A value of 0 means that the item being described is the item at the URL. A 
value of 2 means the URL is for a related resource. If the second indicator is 2, there nxist be a 
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corresponding 13 to explain the relationship. To avoid mistakes in the URL, whenever possible, one should 
use copy and paste to enter the URL information. 

856 40 lu httt>://www. nasa.gov/home/index.html 

856 42 BSpanish version lu http://www. nasa.gov/about/hiuhlights/En Espanol.html 
Let us look at one part of the NASA site and what that surrogate record would look like. Suppose 
that every year, Mrs. Brown had her students investigate the planets. A site on the planet Mars would be 
particularly helpful, especially if the source was as authoritative as NASA. Figure 3 shows the front page of 
the Mars site and below is the full record. Notice the extended 245 tag: we must account for the parent site 
as well as the titles for the site itself. One of the problems with cataloging a part of a web site is deciding 
how to deal with the separate title. This is easily accommodated by using the Ip for the part of the title. 
Notice too that a corporate entry has been added in the 710 tag for the Jet Propulsion Laboratory. Since JPL 
is part of the URL, the cataloger decided to make an entry for that entity. 

245 00 laNational Aeronautics and Space Administration lh[electronic resource] :lbsolar 
system exploration. IpPlanets. IpMars. 

246 30 laSolar system exploration 
246 30 laPlanets 

246 30 laMars 

260 _ laWashington, D.C. rlbNASA. 

538 laMode of access: Internet. 

500 laTitle taken from home page (viewed Oct. 13, 2004; last updated Jan. 9, 2004) 

520 1_ laPart of the Planets section of the NASA web site, this page focuses on the planet 
Mars with links to the Mars exploration and photographs. 

505 0_ laOverview - Moons - Gallery- Facts and figures - Kid’s eye view - Resources. 
610 10 laUnited States. IbNational Aeronautics and Space Administration. 

650 _0 laMars (Planet) 

650 _0 laPlanets 
650 _0 laSolar system. 

650 _0 laAstronomy. 

710 2_ laJet Propulsion Laboratory (U.S.) 

856 40 lahttp://solarsystem.jpl.nasa. gov/planets/profile. cfm?Object=Mars 

Conclusion 

We have seen that it is not really so tough to catalog a web site. Indeed the benefits of having 
students rely mostly on the OP AC for information access truly outweighs the beginning difficulties of 
getting used to a new cataloging template. There are some vendors who offer monitoring programs for 
schools with sites on their catalogs. Programs such as Follett's "Monitor" allow the librarian to register sites 
with Follett. Follett then monitors the site and notifies the librarian of any URL changes. 

It would be great if one day a student could type in “skull” and be lead to books, Internet sites, and 
database articles. Until such a time, the least we can do is connect our books with good Internet sites. The 
way to do that is by cataloging the sites that we already have bookmarked. Whether one calls this 
cataloging metadata or web sites, the point is that the job gets done and that we pull our students out of the 
Internet Ocean and back to the library catalog for one stop information access. 
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Literacy defined 

Since the early 1900s, teaching literacy has always been like trying to hit a moving target. The 
only consistency has been that society has tended to group individuals into two broad categories of being 
literate or being illiterate based solely on whether or not they could read or write (NCREL & Metri Group, 

2003) . The ability to read, write, listen, and speak have always been a corner-stone in these literacy efforts 
in an attempt to broadly serve the interests of formal education. Because of outside pressures to increase 
literacy under these terms, educators have generally focused their literacy education efforts on teaching 
children how to code and decode words. But over time, the focus of literacy instruction has morphed to 
reflect changing economic times, social developments, and historical events (Lankshear & Knobel, 2003; 
Tyner, 1998). By 1970, the definition of literacy extended beyond merely decoding and encoding printed 
words to include the ability to reflect on and analyze the world (Lankshear & Knobel, 2003). A further 
major expansion of the term literate took place in the 1990s. In 1991, the National Literacy Act was passed 
that was designed to ensure that adults have all the skills necessary to function effectively in the work place 
and at home. Congress (Sec. 3) defined the term literacy as “...an individual's ability to read, write, and 
speak in English, and compute and solve problems at levels of proficiency necessary to function on the job 
and in society, to achieve one's goals, and develop one's knowledge and potential’'. However, in 2002, 
President George W. Bush, in an effort to re -focus literacy back to reading and writing, included a Reading 
First initiative in the No Child Left Behind Act. Reading First is designed to ensure that children receive 
effective reading instruction in grades K-3 by providing additional help to state and local school districts 
and to establish high-quality comprehensive reading instruction programs that include the five so-called 
major components of reading: phonics, phonemic awareness, vocabulary, fluency, and comprehension. 

However, due to the realities of the world we live in, the real definition of literacy has yet again 
swung away from a very narrow definition and has begun to reflect the technological age in which today’s 
students live. The digital age has thrust our nation’s youth into a highly advanced, technological, global 
society. With these advances have also come increased demands on what and how students are expected to 
learn. In addition to basic literacy, students are expected to attain proficiency in scientific, economic, 
technological, visual, informational, and multicultural literacy (NCREL & Metri Group, 2003). 
Unfortunately, all these re -focusing initiatives and increases in funding appear to have come up short and 
have yet to realize any significant positive outcomes. The National Assessment of Education Progress’ 
(NAEP) reading report card has not only shown very little change in the reading performance of fourth 
graders since 1992, but also a decrease in performance of eighth graders. The 2003 report showed that 37% 
of fourth graders were reading below the basic level as compared to 38% in 1992. With regards to eighth 
graders, the report indicated that in 1992, 31% were below the basic level and in 2003 the percentage 
dropped to 26%. More recent reports show that fourth grade reading gains have been sustained while eighth 
grade performance has been inconsistent. In 2002, the amount of eighth grade students below the basic 
reading level was 25% which slightly increased in 2003 to 26% (National Center for Education Statistics, 

2004) . 

Regardless of the definition of literacy that is in mode at the time, there has also always been a 
negative stigma associated with being illiterate (Withrow, 2004). Educators have somehow misguidedly 
assumed that literacy is tied to intelligence. While some narrow definitions of literacy might corroborate 
this assumption, a holistic view of intelligence reveals that tying literacy to intelligence can result in a 
mischaracterization of a person’s abilities. Pointing out that everyone has their strengths and weaknesses, 
Corcoran (1981) defined intelligence as a skill in a particular medium, suggesting that the symbolic codes 
used in any medium that serves a communication purpose are internalized by those knowledgeable in that 
medium and, therefore, become authentic tools of thought. In other words if one can communicate well in 
one medium but not another, s/he shouldn’t be generally classified as unintelligent, but, rather, just 
unskilled in that medium. S/he might be very bright or clever in another medium if that medium is one that 
s/he uses most of the time. 
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Corcoran’s ideas generally concur with McLuhan’ s the medium is the message concepts (1964), as 
well as more recent educators like Mitchell Stephens (1996) and Marc Prensky (2002) who believe today’s 
youth are developing a literacy of their own kind based on new forms of communicative media. What this 
means is that perhaps we are selling today’s youth short with broad-brush labels like ‘stupid’ or ‘ignorant’ 
simply because they don’t interact well with text -based media. The problem with such an incorrect 
diagnosis is that educators may be applying the wrong specific treatments for the wrong problems, or they 
are simply relying on broad spectrum antidotes that are hit-or-miss because we simply don’t understand the 
environment in which our children live. The disappointing results of NAEP report card certainly bears this 
out. While these results are certainly substandard as they relate to what they are saying about our children’s 
abilities to read and write, perhaps everyone is overreacting and making some incorrect assumptions about 
the general overall intelligence of our youth. If McLuhan was correct in saying that the media people use 
not only define the message but often also define those using that media, then we should be able to learn 
what it is different about how our children communicate with one another. Perhaps it is time to take the 
time to understand the communicative skills they might actually possess and use them in our instructional 
methods and strategies. 



Growing up digital 

Most would agree that they probably have noticed that today’s youth use computers and other 
visual media a lot. However, the actual statistics reveal some astonishing results. As many as sixty-five 
percent of children in the United States are already online prior to their teen years -many even as young as 
two years of age. The U.S. Department of Commerce estimates the current growth rate for Internet use at 2 
million new users per month; the majority of which are children and teens (NCREL & Metri Group, 2003). 
Today’s youth are being inundated with technologies that are allowing them to actively participate and 
communicate in a world of their own with a variety of new forms of media (Serim, 2003). It is most 
revealing to note that, while we label these advances as technology and new media, children view these new 
forms as neither being technological , nor new. As Alan Kay once said, technology is only considered to be 
‘technology’ if it was invented after you were born (Tapscott, 1998). 

The use of computers, videos, DVDs, and television are popular forms of digital media that have 
long become a part of education. They have been found to be effective because they combine moving 
pictures and audio, and have the ability to appeal to a variety of learning styles (Honey, Pasnik, Saltrick, 
2004). What is different today than with previous generations of technology integration is that the 
democratization of the technology has advanced to the point that there is an ever-increasing use of digital 
media in leisure time activities. This ever-increasing daily use has drastically changed the way children 
think, learn, and how they give and receive information (Dresang & McClelland, 1999). Before students 
enter kindergarten, they are becoming well-accustomed to non-linear technologies such as CDs, DVDs, and 
the Internet that allow them to choose their learning paths. Students entering school today do not know of a 
world that does not include the Internet. In a study conducted by the Kaiser Foundation researchers 
reported that children ages zero to six spend an average of two hours per day using screen media which 
includes television, videos/DVD, computers, and video games. On the other hand, they only spend an 
average of 39 minutes a day reading or being read to by others. Among that same age group (0-6), 36% of 
children have televisions in their bedroom, 27% have a VCR/DVD player, 7% have a computer, and 3% 
have Internet access in their bedrooms (Rideout, Vandewater, & Wartella 2003). Studies have supported 
researchers’ claims that children spend an average of four thousand hours over their teenaged years in front 
of video or computer screens (Greenfield, 1984; Healy, 1998; Tapscott, 1998; Prensky, 2001). 

Some have attempted to tie computer usage to increases in ADD/ADHD and/or violent behaviors 
(Ernes, 1997). While no causal effect has been shown between computer usage and shortened attention 
spans, this disparity in how much time teens spend on digital versus traditional media has to be doing 
something with their brains. Trying to determine a direct causal relationship between apparent shortened 
attention spans and television viewing is a perfect example of the so-called “chicken and egg’’ dilemma. 
Have the shorter segments become more prominent because of viewer preferences, or have the shorter 
segments actually re-wired our children’s brains? Several cognitive scientists have opted for the latter. 
Several studies have supported claims that today’s teenagers’ brains have been, somehow, rewired by this 
increased exposure to computerized media (Tapscott, 1998; Fiore, 1997; Diamond, 1988; Goode, 2000; 
Healy, 1998; Moore, 1997). Whether or not everyone can agree to the extent this has happened, there is no 
question that today’s media -centric youths somehow perceive and learn differently than previous 
generations (Prensky, 2001). Further, Prensky and others like Mitch Stephens (Stephens 1996) make the 
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case that today’s “games generation” has such familiarity in the digital domain that ‘digital’ has become so 
ingrained into their being that it has become their native and primary form of communication. The so-called 
generation gap is more likely one of differences in the choice of preferred communication modes rather 
than one of age: media centric youths speak digital. Others who travel outside of this domain are mere 
digital "immigrants’ (Rushkoff, 1997; Prensky, 2002). 

We educators need to take heed. If we properly analyze the effects these new participation and 
communication paradigms, we should soon realize that the so-called digital media revolution even further 
extends what it means to be literate. More importantly, a proper perspective can lead to revolutionary 
thinking as to how children can be motivated to acquire the traditional letteracy skills (i.e., skills in text- 
based media) that haven’t gone out of style because of their importance in the world they will move into as 
they grow older. We simply need to realize that our children speak ‘digital’ and, to them, text -based 
communication is like learning a second language. The difficulties we face in motivating them towards the 
text -based world are similar to those we used to have in motivating students to speak foreign languages. 
(Recall your own school days when you questioned the need to learn another language when everyone 
around you spoke English). On the other hand, perhaps we could learn a lot about how to go about teaching 
traditional literacy skills if we utilized some of the powerful teaching and learning techniques s found in 
today’s ESOL classes. 



Digital media for reluctant readers 

We could also make tremendous advances if we combine what we know about teaching second 
languages and also take lessons from the skills being developed in digital media movement found in 
programs being developed in major universities all across this country and internationally. These programs 
are growing exponentially due to the demand placed on them by the younger media-centric generation. 
Many of the digital media courses center on various forms of art, music, graphic design, television, and 
entertainment. The constant thread in all of these concentrations is that, regardless of the media used, at its 
core is the idea of story. The most concise definition of the term digital media, as it is currently being used, 
includes the concept of convergence of arts and technology for the purposes of creating multiple forms of 
human expression and communication. If one accepts that definition, then the term literacy would again 
change to include skills communicating via these new digitally mediated channels. Because story is at the 
core of both traditional literacy and digital media, using it as a focus should be very attractive to our digital 
media -centric youths. Interestingly enough, many of the techniques taught in digital story-telling include 
things like immersion and total physical response -some of the same successful techniques found in second 
language learning classes. 

These changes in perceptual, cognitive, and communicative styles bring up several interesting 
questions with regards to the kinds of mediated instructional strategies that might motivate reluctant readers 
in today’s media -centric society. According Diana Kimpton (2004), there are two different types of 
reluctant readers -those who can read but don't enjoy it and those who find reading so difficult that they 
avoid it whenever they can. Both groups think reading is hard work and is anathema to members of a 
media-centric culture who often feel like immigrants in a literate society. Motivating reluctant readers in 
this new digital age can be a challenging task. Students are motivated by different types of inputs and 
supplemental activities. There is considerable research that links motivation to the effect on the person his 
or her past learning experiences, one’s assessment of self-efficacy, attitudes and perceptions (Keller 1983). 
Knowledge of the fact that ‘digital’ is the preferred natural language of the media culture may be of some 
help leading the way to possible solutions to the literacy problem. Using the theory of teaching to one's 
strengths and remediating the differences (Doman 1984), a properly constructed multimedia tool could be a 
way to attract the attention of otherwise reluctant readers. Given that one agrees with those who believe 
that most personal experiences for today’s “games generation” are more than likely digitally mediated in 
one form or another, then it is not too difficult to also believe that digital mediation should have a positive 
affect on literacy skills development, including attracting reluctant readers. 

Matching books with potential readers 

There have already been several instances of successes using mediated tools in teaching and 
learning in which students have been shown to improve word recognition, reading comprehension, and 
spelling skills and to boost reading scores and self-esteem (Taylor, Hasselbring, & Williams, 2001). In 
recent years, computerized reading incentive programs have increasingly become apart of school literacy 
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curricula despite some debate over their validity. Opponents of these programs argue that the external 
motivation to read created by the system of rewards will fade once rewards are withdrawn (Biggers, 2001). 
But program sponsors like Scholastic ( Reading Counts /) and Renaissance Learning ( Accelerated Reader) 
claim that they have had successes in motivating students to read by using this system of rewards (Engwall, 
1999). Regardless if one agrees with the alleged positive effect of these reward programs, classroom 
teachers have begun to recognize what librarians and media specialists have known for years -if you 
properly match potential readers to an author or genre, even the most reluctant will be more likely to 
complete the book and to read others from that favorite author or genre (Eriksson 2002) . Even though the 
bulk of his findings actually discredited much of these claims, Stephen Krashen (2002) in his comparative 
analysis to determine what aspects of reading incentive programs accounted for their reported success, 
noted that increased availability of high interest books and opportunity for sustained reading that were 
provided to students with access to the programs accounted for the lion’s share of increased learning 
achievement. Playing match- maker is harder than it would seem. Other than the design and/or limited 
contents found on book jackets, there is very little that potential readers can use to identify books that they 
might want to read. Media specialists and teachers often find themselves being asked by students to make 
book selections for them. Better educators have resorted to creating a series of questions to ask the students 
in order to identify their interests. The questionnaires include such things as favorite movies, hobbies and 
things to do, reading level, etc. Even the most probing of questions and not infallible and there is also the 
risk that incorrect recommendations result in students not liking the suggested books and, therefore, not 
completely reading them. Several wrong selections can result in even more readers becoming reluctant and 
being turned off to reading. 

A strategy that has had some successes in helping to match potential readers with books is the 
booktalk. Aidan Chambers, an author of children’s books and a literature teacher, has published several 
works advising how to encourage children to verbalize their literary experiences. In 1985, he coined the 
term booktalk that identified the concept of talking about reading in reader-response contexts (Chambers, 
1985). To Chambers (1993), talk about books is an essential part of a reading selection strategy, which 
includes book stock, availability, accessibility and presentation. Unknowingly, he, too, has borrowed from 
the total physical response and immersion concepts found in second language learning. Others have further 
extended their understanding of the language of their audience and have added new media metaphors to the 
mix. On her website, Nancy Keane (2004) explains that the purpose of a booktalk is to sell the book and to 
grab the audience's interest and make them want to read the book. She even goes so far as to liken a 
booktalk to a movie trailer. Others (Young, 2001) propose that actual movie trailers or short scene 
selections from the movies made from the books can be used as an aide. 

Which comes first, the movie or the book? 

As we have seen, our children live and communicate in a visual world. While some would argue 
that watching the movie first might ruin the intellectual experience found in exercising one’s imagination 
while reading, there has been some experimental research that would support the notion that seeing the 
movie first might help the reader better understand and comprehend what s/he is reading (Gropper 1966; 
Nugent 1982). The question, then, is how to provide media-related visual pre -reading organizers without 
spoiling the reading experience. It would seem, therefore, that using digital versions of booktalks and 
trailers would be an interesting middle way in the pursuit of these match-making activities. Just as movie 
trailers have been very successful in influencing audiences in selecting the movies they watch, it would 
seem logical that trailers made specifically for the books would do the same for books. These trailers, in 
essence, might be considered an animated book jacket and need to be created in such a way to both attract 
potential readers and to crystallize the essence of the context of the books, using visualizations of the 
characters, themes, and metaphors, etc. Putting these trailers online so that they may be easily accessed via 
the Internet makes sense. 



Digital Booktalk 

Digital Booktalk (DBT) (http://www.digitalbooktalk.com) is an online portal on which several of 
the previously successful strategies are used to aide potential readers to evaluate whether they would like to 
read specific books. The trailers are one of a series of activities found on the website that aid in the book 
selection process. The number of titles of books on the site is growing and follows school recommended 
reading lists and those found on the rewards programs like Accelerated Reader and Reading Counts. Using 
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automated intelligence, the suggest-a-book feature replicates the ‘interest questionnaires” that generally 
take place between prospective readers and librarians, media specialists, and teachers. DBT employs an 
avatar to posit several questions regarding a reader’s interests in things like movie genres, previously read 
books, hobbies, and reading level (if known). A list of suggested trailer titles is then presented based on 
matched results from its database. Students can then browse through the suggested trailers that are 
prioritized according to matching percentages and then check off the books that they think they might be 
interested in as they review the list. A user profile keeps track of the results of the questionnaire (especially 
the question concerning previous books they may have read and movies they particularly like) so that in 
future sessions the system can remind them of their previous choices. In addition, utilizing a technique 
similar to what other commercial online booksellers use, visitors to the site are also reminded of similar 
books that others have selected with similar interests (people who have read so-and-so book also have 
read...). 

The third aspect of the DBT site is a supplemental section in which interactive activities act to 
provide further background on the context of the books and generally aide in helping students make their 
selection. For example this section includes a digital version of one or more of the activities suggested by 
widely publicized ‘booktalkers’ like Nancy Keane (2004), including word searches and brain teasers to 
help build pre -reading vocabulary. Another activity is a “who am I” game in which visitors are asked to 
guess the main characters in books whose identities are progressively revealed as interact with things like 
taking the visitor on a tour of the geographic settings that are contained in the book. The intent of these 
character development games is similar to recent marketing tie-ins between Pixar studios and marketers 
like Proctor and Gamble, McDonalds and others (Howard, 2004) in which characters from the animated 
film The Incredibles are introduced to the public before the movie is even released. Viewers already have a 
sense of identity with the characters when they attend the movie, and their enjoyment of the storyline has 
been preserved. In similar fashion, these pre-reading activities provide the same modicum of familiarity 
and readiness for the reading activity developed through the limited back-story information that is revealed 
while playing the games. Character identification is limited in that it only reveals enough memorable 
information about the characters so that the reader will become familiar with them but not so much that it 
spoils the discovery process that takes place while reading the books. The characters are identified only to 
the extent that helps to set the scene and to provide a small preview of the background or point of view of 
the storyline, similar to what is done in story circles and live booktalks. 

UB the director 

Given the nature of today’s students, it isn’t surprising that watching a movie is often preferred to 
reading a book. Teachers are often faced with the inevitable question as to why they need to read the book 
rather than watching the movie made from it. Most teachers would rather provide class time to watch the 
movie as a reward for reading the book in spite of the fact that there is some research that seems to indicate 
that many students might actually do better to watch the movie first (Gropper, 1966; Nugent, 1982). In 
spite of their strong opinions on whether watching the movie first ruins the opportunity to stimulate 
reader’s imaginations, teachers struggle with how to construct a believable and appropriate answer to the 
question. One way to answer this question in a positive manner is to reply by reminding the students that a 
movie is the result of someone else deciding what goes in it. Not all movies remain true to the book, and 
besides wouldn’t they like to be the director of their own movie about the book? The planting the idea with 
students of reading the book as if they are going to make a movie out of it is a potentially positive way to 
motivate and reinforce the concepts of visualizing while reading so important to teaching literacy skills. 

The problem is that there isn’t enough time in the classroom for each student to make a full-length motion 
picture from the books the students read. Acting like a booktalk that has already been shown to enhance 
perception through personal actions ((Neuman 1990), a two minute movie (book) trailer about the book is a 
much easier final product. In this case, the student producer has to know enough of the details about the 
setting and context and needs to make some particularly insightful decisions as to which scenes need to be 
put into the trailer, providing a reason for him or her to be reading the book critically for content and 
context. 

A UB the Director section of the DBT portal is reserved for these student-produced trailers. The 
submitted trailers are peer-reviewed by volunteer teachers and student groups. This section also contains a 
lesson plan with references on how to produce the trailer, and suggestions for the students and teachers as 
to what to look for in the books they read. Student-produced trailers are co-mingled on the resulting lists 
along with the professionally produced trailers but are notated as such. 
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Future research -collecting empirical data 

There is enough anecdotal evidence to suggest that the DBT portal has resulted in students 
becoming familiar with the books before they read them. However, because it is to be used in an 
educational environment, actual empirical data needs to be studied. It isn’t enough to help students become 
familiar with the titles, a determination will be done in the upcoming months to find out if increased 
knowledge translates into interest in the books and, therefore, results in actual increases in reading and 
completing the books they are matched with. The study will focus on the two levels. First, on a practical 
level, a determination will be made to find out if the portal actually helps students select books and in such 
a way that increases the number of books they actually complete. Secondly, from a pedagogical standpoint, 
do the activities and trailers encourage students to critically analyze, reflect upon, and write about their 
selected book throughout reading In other words, does the portal effectively increase the values associated 
with the traditional concepts of literacy ? 

Besides these educational issues, there is the further question of which books need to be 
represented on DBT. The ultimate domain of books that need to be included on any reading list is 
extensive, to include a number somewhere between one book and the total number catalogued by the 
Library of Congress. In reality, the initial implementation will be aimed at two groups in particular: those in 
the fourth grade and those in the eighth and ninth grades. It is these two groups that were studied by the 
National Assessment of Education Progress (NAEP) report card. It is also these two groups that are the 
focus of most statewide efforts are being exerted to improve standardized test scores. Using Reading 
Counts! and Accelerated Reader as well as school-suggested reading lists as the starting point, the number 
of candidates for the trailers totals somewhere between one and three thousand. Prioritizing the list from a 
large number of sources will reveal that some books show up more often than others. It is these commonly 
used books (like To Kill a Mockingbird, or Animal Farm, among others) that will be the first trailers to be 
developed. The studies will be constructed in such a way that the power of the data analyses will be derived 
by the construct of the comparisons and the number of students who participate, rather than the volume of 
books represented. 



Summary 

Educators, have focused their literacy education efforts on teaching children to code and decode 
words. In spite of all the various efforts to teach literacy to our children; the various report cards indicate 
that, perhaps, a new way of doing teaching literacy should be investigated. Today’s media -centric youths 
would benefit from using mediated programs to help motivated and talk to them in their own language. 
Several instances of successes using mediated tools for teaching and learning in which students have been 
shown to improve word recognition, reading comprehension, and spelling skills and to boost reading scores 
and self-esteem. 

Digital Booktalk is a web portal that uses mediation to extend the techniques that have already 
proven successful in increasing children’s interest in reading and understanding books. In its four sections, 
students are given the opportunity to match their interests with book titles, see trailers that help them 
visually organize the contexts from the books, and participate in learning activities that provide initial 
insights to vocabulary, characters, and themes. In addition, students are given the opportunity to create 
book trailers on their own with the possibility of having them published on the portal. It is believed that 
these types of activities will encourage students to correctly select books to read for completion and also 
find it easier to make future selections. Research on the effectiveness of this portal are planned to collect 
and analyze empirical data. Future enhancements can be made to the project based on the results of this 
research. 
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Abstract 

Only few studies have explicitly attended to the nature of the perceived underlying factors that 
prompt young adolescents’ reflective thinking in association with K-12 learning environment. This paper 
focuses on an analysis of the factors that are perceived by young students as prompting their reflective 
thinking and how those factors apply to the practice of design. Given that technology can provide 
scaffolding for reflection ( Lin,X., Hmelo, C., Kinzer K., Secules, T 1999) and given that without 
appropriate support students have difficulty engaging in high-level reflective thinking ( Hmelo, C. E. & Lin, 
X. 2000), it is important to look at the design issues and elements that should be considered in supportive 
learning environment design (focused on K-12 learning environment). This paper examines and analyzes 
learner perceptions of the role of the learning environment elements including cognitive and affective 
scaffolding, instructional strategies, and tools as important in supporting their thinking. The factor analysis 
revealed seven dimensions of helpful cluster of learning environment elements. Four distinct dimensions of 
them are such as feasibility of other’s supportiveness, flexibility, cognitive scaffolding and learner’s 
independence. The most helpful factor was the feasibility of other’s supportiveness with the most helpful 
clusters of elements as having other’s help and teacher’s individual caring and encouragement. 
Recommendations are provided for designing learning environments that prompt reflective thinking based 
on these results. 



Introduction 

The explosive growth and development of technology requires new knowledge and learning skills 
(Lin, Hmelo, Kinzer, & Secules, 1999). This explosion is reforming the learning environments and 
educational concepts. In particular, the growth of technology that includes the World Wide Web requires 
students to learn not only how to use resources to find relevant information but also how to make sense of 
information. Higher-order thinking skill is especially important when deciding which sources are useful 
and reliable (Lin et al, 1999). Therefore, in this new learning paradigm something other than “just thinking” 
is involved. In other words, it requires learners to be able to think reflectively and critically to manage 
information accessible via this new learning environment 

With regards to the K-12 learning environment, it was discovered that young students perceived 
the learning environment as one of the most important factors that prompt and support their reflective 
thinking (Koszalka, Song, & Grabowski, 2002). In their research context, learning environments referred to 
the flexible learning climate where students have freedom in class, work with a partner, and have time to 
think. The use of the term “learning environment” is a narrow scope. In the current study, “learning 
environment” refers to the term more broadly as it is associated with K-12 classroom learning 
environments. It adds teachers ’ teaching strategy, supporting tools, learning climate, characteristics of 
learner-centered classroom, peers, and technology support. As to the student perception of their learning 
environment, there is research that revealed the importance of studying students ’ perceptions of classroom 
practices by illustrating the influence of student perceptions of personal climate in class on their motivation 
and classroom performance (Meece, 2003). According to Meece (2003), students’ perceptions of classroom 
practices including teacher’s teaching and students ’ learning atmosphere were predictive of student 
motivation and achievement. These findings, therefore, emphasize the importance of taking into account 
students ’ perceptions of teaching practice as well as the elements of a more narrowly defined learning 
environment. 

From this perspective, it is important to look at the elements that should be considered in 
designing a supportive learning environment for students’ reflective thinking. These include different 
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teaching strategies and scaffoldings, tasks, reflective thinking tools, atmosphere of learning environments, 
and technology-enriched classroom learning environments that can be provided in K-12 education. Taken 
together, as a part of this research, the student perception of their learning environments that prompt and 
support their reflective thinking was examined, which is important for teachers and instructional designers 
to apply to their design practice. In short, this study will examine and analyze both learner perceptions of 
the factors as important in prompting their reflective thinking and which characteristics of a learner- 
centered classroom are most important in supporting young -students’ reflective thinking. 

Theoretical Background 

Factors Prompting Students’ Reflective Thinking in K-12 education 

Reflective thinking is defined by Dewey (1933): “Active, persistent, and careful consideration of 
any belief or supposed form of knowledge in the light of the grounds that support it and the further 
conclusion to which it tends.”, and it is more relevant for ill-structured problems (King & Kitchener, 1994). 
In other words, reflective thinking is viewed as a thinking process reflecting specifically on the processes 
about what has happened. This process leads students to assess what they know, what they need to know 
and how they would learn the information to make judgments to construct learner’s own knowledge. 

When it comes to fostering reflective thinking in K-12 education, such other potentially important 
factors as teacher’s teaching strategies and scaffoldings in class (i.e: question prompt, instructional 
strategies, and so on), tasks (i.e: making observations, asking question, and comparing their 
understandings with those posed by others, etc.) and tools (i.e: discussion, tutoring, and peer evaluation, 
etc.), and learning atmosphere (i.e: having freedom, time to think, etc.) must be considered. ( Douillard, 
2002; Koszalka et al, 2002; Wood, Bruner, and Ross, 1976). 

Teaching Strategies and scaffolding : The role of the teacher in scaffolding students’ learning is 
crucial to the development of reflective thinking not only while the students are engaged in the task but also 
in the creation of a learning environment that encourages young children to actively play and/or explore the 
objects and ideas that they encounter (Yelland, 1999). 

Teachers can prompt students’ reflective thinking by asking reflective questions as well as 
explaining reflective concepts to students (Moon, 1999; Virtannen et al, 1999). Asking questions that seek 
reasons and evidence can prompt students’ reflection, and providing some explanations to guide students’ 
thought processes during explorations can provide an opportunity to support their reflective thinking 
( www.higp.hawaii.edu/kaams/resource/reflection.htni) . 

Tasks: In class, when students are provided with authentic tasks involving ill-structured learning 
activities, challenging work and complex problem solving, they are prompted to use reflective thinking 
during learning (Hopson, Simms, & Knezek, 2001; Stepien & Pyke, 1997). John Dewey (1933) and King 
and Kitchener (1994) propose that individuals engage in reflective thinking when they encounter problems 
with uncertain answers, when no authority figure has an answer, when they believe no one answer is 
correct, and when the solution cannot be derived by formal logic. 

In addition, the use of reflective thinking can be pronpted while students are engaged in active 
learning, inquiry and problem-solving (Yelland, 1999). Sternberg (1985) also addressed that the use of 
higher order thinking skills and the use of metastrategic processes which requires students’ reflective 
thinking are characteristics of effective problem-solving environments. 

Yelland (1999) noted some characteristics of those learning activities: 

• is authentic and therefore meaningful, 

• is interesting and engaging, 

• allows for initiative, 

• can be discussed and summarized into statements that have meaning for students -allows 
the student to use what knowledge he or she knows and then explore new 

• concepts, and 

• does not have one path solution or has a variety of acceptable solutions. 

Reflective Thinking Toolsi The types of tools that scaffold students’ reflective thinking such as 

reflective journal writing, guiding questions, and concept maps are important in fostering students’ 
reflective thinking ( Kinchin & Hay, 2000; in Koszalka et.al, 2002). There is a shared assumption of 
educators that reflective writing can promote reflective thinking (King & Kitchener, 1994; Ross, 1990). 
Providing students with keeping reflective journals to write down students' positions, give reasons to 
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support what they think, show awareness of opposing positions and the weaknesses of their own positions 
can foster students’ reflective thinking ( www.higp.hawaii.edu/kaams/resource/reflection.htrri) . 

Learning Atmosphere : Learning atmosphere in which students engage in can be an important 
factor prompting their reflective thinking. Research shows that students perceive the student learning 
environment factor that includes flexibility as the most significant factor to help think reflectively 
(Koszalka, et al. 2002). In terms of K-12 education, the quality of the classroom climate even can 
influence their levels of achievement. According to Poplin and Weeres (1992), their research sums up this 
issue: “when students feel alienated and disconnected from the process of learning and from the social 
context of learning, levels of achievement are lowered” (p. 15). 

A Classroom as a Learner- Centered Learning Environment 

the perspective that couples a focus on individual learners (their heredity, experiences, perspectives, 

backgrounds, talents, interests , capacities, and needs) with a focus on learning (the best available 
knowledge about learning and how it occurs and about teaching practices that are most effective in 

promoting the highest levels of motivation, learning, and achievement for all learners) The 

learner-centered perspective is a reflection of the twelve learner-centered psychological principles 
in the programs, practices, policies, and people that support learning for all. (p. 9) 

Learner-centered learning environment : One very powerful type of classroom that supports 
reflective thinking is the learner-centered learning environment. Learner-centered attributes such as a more 
flexible atmosphere, time, and tasks, and peer tutoring activities are included in the learning environment 
design factors (Song, et al. 2002). In particular, when these design elements are associated with designing a 
learner-centered learning environment, they are perceived as important factors promoting students’ 
reflective thinking by young students (Song, et al. 2002). Based on these research findings, there are some 
suggestions made for designing a classroom as a learning environment that supports and scaffolds students’ 
reflective thinking. For instance, either teachers or instructional designers are recommended to take into 
account some flexible and affective scaffolding: providing enough wait-time for students to reflect when 
responding to inquiries, providing emotionally supportive environments in the classroom encouraging 
reevaluation of conclusion and providing students less-structured learning environment that prompts 
students to explore what they think is important 
( www.higp.hawaii.edu/kaams/resource/reflection.htm ). 

Interestingly enough, many of these design elements that are regarded as supporting reflective 
thinking environments overlap with the key characteristics of the learner-centered framework. Therefore, it 
is valuable to examine the design elements that are potentially influential to learners’ perception of 
important factors prompting their reflective thinking. The following is a review of recent research on 
learner-centered classrooms, learner-centered psychological principles, and learning environment with both 
technological and non-technological support. 

Importance of Learner-Centered Practices for students’ reflective thinking in A’-72:Learner- 
centered practices involve caring, establishing higher order thinking, honoring student voices, and adapting 
instruction to individual needs. Research findings that used survey data from 2,200 middle school students 
across the United States indicate many important benefits of learner-centered practices for young students: 
more positive forms of motivation and greater academic engagement (Meece, 2003). This implies the 
importance of learner-centered practices for young students’ reflective thinking. 

Based on the literature of reflective thinking and learner-centered learning environment, this study 
examined the following research questions: 

1. What are the factors that are perceived by young students as prompting their 
reflective thinking? 

2. Which characteristics of a learner-centered classroom are perceived as most 
important in supporting young-students’ reflective thinking? 

Method 

Subject 

A survey was administered to three-hundred fifty three sixth grade students attending an 
elementary school located in Korea. Students were from 9 different classrooms; including 199 boys, and 
154 girls. 
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Instrument 

The survey was constructed according to the rationale that students perceive elements related to 
their learning environments as important in prompting their reflective thinking (Song et. al, 2002). These 
attributes are composed of such elements as teachers ’ teaching strategies, peers, tools, and classroom 
climate. In particular, when these elements are present in the learner-centered learning environments, 
students perceived them as very important Since this study takes the assumption that the learner- centered 
practices will affect students ’ reflective thinking, 12 learning environment elements associated with 
learner-centered classroom practice McCombs, & Whisler (1997) suggested were added. 

The survey for measuring the perceived factors related to reflective thinking was adapted from 
related research and developed by the author based on the literature of reflective thinking and learner- 
centered learning environment. The original instrument contained 10 items (Koszalka et al., 2002). Twelve 
more items relevant to the characteristics of students’ learning environment were added. The final 
instrument consisted of 22 items that are scored on a 5 -point Likert scale from strongly disagree (1) to 
strongly agree (5). The survey was reviewed for construct validity. The internal consistency reliability 
coefficients, Cronbach’s alpha, for this version of the instrument that includes 22 items was .82 (n=353). In 
addition to internal consistency reliability coefficients, Table 3.3 contains estimates of the test-retest 
reliability of the questionnaire. These students responded to the questionnaire a second time approximately 
two weeks after the first administration. These calculations were based on data from a subsample of 38 
sixth grade students in the sample school. The correlation (r) between the two tests computed from the test- 
retest correlations were also measured. The test and retest values computed from responses to the 
questionnaires were highly correlated (r = 0.80, P < 0.0001), thus indicating that the survey questionnaire 
displayed quite good test-retest reliability. 

Data Analysis Procedure 

The programmed SPSS-X package was used to process the data. The data were analyzed to answer 
the sub-questions of the problem statement. A principal component analysis with a varimax rotation was 
adopted to determine distinct learning environment element dimensions among the 22 items. Factor 
loadings were rotated using varimax rotation to identify the simple structure of the survey. Factor scores 
were derived. Pairwise deletion was used in the analysis of each survey to compensate for missing data. 
Reliability of the factor dimensions was computed through the reliability procedure in SPSS-X. The 
Cronbach alpha coefficient was referenced to assess level of reliability. 

Results 

Research Question 1 : What are the factors that are perceived by young students as prompting their 
reflective thinking? 

In order to examine whether there were distinct learning environment factors that are perceived by 
sixth grade students as prompting their reflective thinking in Korea, a factor analysis approach was 
adopted. A principal component analysis with a varimax rotation was adopted to determine distinct factors 
prompting reflective thinking among 22 statements. This resulted in seven factors with an eigenvalue 
greater than one. Items with a factor loading of at least .40 were selected for each factor. Two items (Item 
No. 3, No. 16) loading of less than .40 were removed from the groupings. Cumulatively, the seven factors 
accounted for approximately fifty-five percent of the variance in the sample (Table 1). Cronbach Alpha 
coefficients were computed for the items that formed each factor. The coefficient for the seven factors 
ranged from .41 to .64. 

The first factor was titled “Feasibility of Others’ Supportiveness/ adult intervention,” due to 
the statement of which it was comprised. The statements were: 

• having opportunity to work together with others not physically present (e.g: via email/ phone call 
contact) 

• when teacher encourages shared decision making and gives us increasing responsibility for our 
learning 

• when my teacher listens to and respects our points of view 

• when my teacher monitors student progress continually and provides feedback on individual 
growth and progress. 

Factor 1 was deemed most reliable (.64), accounted for 23% of the variance and had an overall mean of 
3.79. 
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Factor two, titled “Flexibility,” loaded three statements: 

• Working on activities in class that have many different answers, 

• Having time to think about a question before answering, and 

• Having freedom in class to explore topics 1 am interested in. 

This factor had a coefficient reliability of . 50, accounted for 6.4% of the variance, and had an overall mean 
of 3.78. 

The third factor, “Cognitive Scaffolding,” was comprised of four statements, they included: 

• drawing pictures to illustrate my understanding of a topic, 

• when my teachers presents us with examples and expects us to generate rules, 

• when my teacher presents us with the rules using pictures, and 

• when my teacher encourages us to formmental connections between concepts . 

A little less important than the previous factor (mean= 3.54), this factor was less reliable (.50) and 
accounted for 5.8 % of the variance. 

The fourth factor, titled “Independence,” included four statements: 

• when my teacher asks me how to solve difficult tasks, 

• working with partners during classroom activities , 

• answering questions about a topic, and 

• working on complex and challenging activities that promote conceptual and analytic thinking. 

This dimension contributed very little (5.3%) to the overall variance, had a mean of 3.62 and was deemed 
reliable with an alpha of .58. 

Factor 5 titled “Alternatives” appeared to be related to alternative approaches of teaching and 
learning was comprised of two statements: 

• when standardized and alternative forms of assessment are used help me think more what I’m 
studying, 

• having opportunity for peer teaching as part of instruction helps me think more what I’ m studying 
A little less important than the flexibility factor (factor 1, M=3.79) and as important as the feasibility factor 
(factor 2, M= 3.74). 

Factor 6 “Reflection process” appeared to relate the scaffolding the process of reflection itself by having 
students get involved with the reflective activity. The questionnaire items included statements as follows: 

• Writing about my understanding of a topic helps me think more what I’m studying 

• Having continuous communications with each other that extends beyond the class lessons helps 
me think more what I’m studying 

This factor had a coefficient reliability of . 45, accounted for 4.9 % of the variance, and had an overall 
mean of 3.38. 

Factor 7 seemed quite different from the previous researches regarding the dimension. According to <The 
APA Learner-Centered Psychological Principles> (1997), which has been categorized into social 
developmental factors. In this study, 

• “working on activities in class related to real problems on earth or in our society” 

element is separated by itself, and it is titled as “Reality” related to the characteristic of the task students 
are involved with. This factor had accounted for 4.9 % of the variance, and had an overall mean of 3.38. 

Table 1. Learning Environments Elements Dimensions Resulting From Factor Analysis 





Learning Environment Elements Statements 


1 


2 


3 


4 


5 


6 


7 


MOTIVATIO 

NALAND 

AFFECTIVE 

FACTORS 

(Feasibility of 

Other’s 

supportivenes 

s) 


14. having opportunity to work together with others not 
physically present (e.g: via email/ phone call contact) 
(n=350) 


.601 


■ 


■ 


■ 


■ 


■ 


■ 


1 8. when teacher encourages shared decision making and 
gives us increasing responsibility for our learning (n=349) 


.610 














19. when my teacher listens to and respects our points of 
view(n=352) 


.705 














20. when my teacher monitors student progress 
continually and provides feedback on individual growth 
and progress (n=350) 


.538 


■ 


■ 


■ 


■ 


■ 


■ 


Overall mean=3.74 

















343 





















MOTIVATIO 

NALAND 

AFFECTIVE 

FACTORS 

(Flexibility) 



COGNITIVE 

AND 

METACOGNI 

TIVE 

FACTORS 

(Cognitive 

scaffolding) 



DEVELOPME 
NTAL AND 
SOCIAL 
FACTORS 
( Independent 
e : learner- 
controlled 
instruction, , 
peer 

collaboration, 

etc.) 



INDIVIDUAL 
DIFFERENCE 
S FACTORS 

Alternatives 



Reflection 

process 



SOCIAL 

FACTORS 

Reality 




Note. 1. Only 



1. Working on activities in class that have many different 
answers (n=353) 



6. Having time to think about a question before answering 
(n=352) 



7. Having freedom in class to explore topics I am 
interested in (n=352) 



Overall mean=3.79 



8. Drawing pictures to illustrate my understanding of a 
topic (n=352) 



1 1 .when my teachers presents us with examples and 
expects us to generate rules (n=352) 



12. when my teacher presents us with the rules using 
pictures (n=352) 



15. when my teacher encourages us to form mental 
connections between concepts (n=348) 



Overall mean=3.55 



4. when my teacher asks me how to solve difficult tasks 
(n=352) 



5. working with partners during classroom activities 
(n=352) 



lO.answering questions about a topic (n=350) 



17. working on complex and challenging activities that 
promote conceptual and analytic thinking (n=352) 



Overall mean= 3.45 



21. When standardized and alternative forms of 
assessment are used (n=350) 



22. Having opportunity for peer teaching as part of 
instruction (n=351) 



O \erall mean=3.74 



9. Writing about my understanding of a topic 



13. Having continuous communications with each other 
that extends beyond the class lessons 



Overall mean= 3.38 



2.Working on activities in class related to real problems 
on earth or in our society (n=353) 



Overall mean= 3.65 



Eigenvalue 



Variance Explained 



Cumulative Variance 



Alpha 



loadings greater than .4 are displayed. 







5.06 


1.40 


1.27 


1.17 


1.13 


1.07 


23.0 


6.36 


5.76 


5.32 


5.10 


4.86 


23.0 


29.4 


35.2 


40.5 


45.6 


50.5 


.65 


.50 


.60 


.58 


.41 


.45 



Research Question 2 Which characteristics of a learner -centered classroom are perceived as most 
important in supporting young -students’ reflective thinking? 

Twelve different learning environment elements that are characterized on a learner-centered 
classroom were listed on the questionnaire. (Refer to Table 2) Sixth grade students were asked to rate the 
helpfulness of the each element they perceive as important. To calculate means and standard deviations for 
the relative helpfulness rating of each learning environment element, category responses was converted to 
numeric values, using a 5-point scale. The closer each mean approximated the value of 5, the higher the 
mean helpfulness utility rating for the learning environment elements. From an examination of the means, 
the following learning environment elements that have learner-centered characteristics were listed from 
“the most” to “the least helpful learning environment elements supporting for their reflective thinking. 
(Table 2) 

In terms of the supportive learning environment elements that are associated with the 
characteristics of a learner-centered classroom, five elements ranked high are related to the 
MOTIVATIONAL AND AFFECTIVE FACTORS such as Factor f , “Feasibility of other’s supportiveness/ 
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adult intervention” and Factor 2, “Flexibility” that includes “allowing different answers, having time to 
think, and having freedom in class to explore topics.” 

Table 2. Learner-Centered Learning Environment Elements 



Item 

No. 


Items reflecting the characteristics of 
the learner-centered classroom 


Factor* 


Mean 


19. 


When my teacher listens to and respects our points of view, it helps me think 
more about what I am studying. 


l 


3.90 


7. 


Having freedom in class to explore topics f am interested in helps me think 
more about what I am studying. 


2 


3.88 


16. 


Having opportunity to create and present whole relationship between what we 
see, hear, or read and what we already know and what we has been learned 
helps me think more about what 1 am studying. 




3.82 


20. 


When my teacher monitors student progress continually and provides feedback 
on individual growth and progress, it helps me think more about what 1 am 
studying. 


1 


3.81 


6. 


Having time to think about a question before answering helps me think more 
about what I am studying. 


2 


3.78 


18. 


When teacher encourages shared decision making and gives us increasing 
responsibility for our learning, it helps me think more about what I am studying. 


1 


3.74 


21. 


When standardized and alternative forms of assessment are used, it helps me 
think more about what I am studying. 


5 


3.74 


22. 


Having opportunity for peer teaching as part of instruction helps me think more 
about what I am studying. 


5 


3.74 


4. 


When my teacher asks me how to solve difficult tasks it helps me think more 
about what I am studying. 


4 


3.58 


14. 


Having opportunity to work together with others not physically present (e.g: via 
email/ phone call contact) helps me think more about what I am studying. 


1 


3.52 


17. 


Working on complex and challenging activities that promote scientific and 
logical thinking in class helps me think more about what I am studying. 


4 


3.31 


13. 


Having continuous communications with each other that extends beyond the 
class lessons helps me think more about what 1 am studying. 


6 


3.30 



*Note: Factor 1= Feasibility of other’s supportiveness, 2= Flexibility, 3= Cognitive scaffolding, 4= 
Independence, 5= Individual differences, 6= Reflection process 



Conclusions and Suggestions for Further Research 

This research study was based on the premise that students can provide valuable information about 
their learning environment elements that prompt their reflective thinking; by listening to students, 
educators, teachers, and instructional designers may be able to enrich the learning environment to support 
students ’ reflective thinking. 

This study identified seven learning environment design factors that a sample of young 
adolescents perceived as helpful in prompting their reflective thinking while studying. The factors 
perceived by sixth grade students in Korea were: MOTIVATIONAL AND AFFECTIVE FACTORS including 
(1) “feasibility of Other’s supportiveness/ Adult intervention” (having opportunity to work together with 
others not physically present, teacher’s affective encouragement), and (2) “flexibility” such as allowing 
different answers, having time to think, and having freedom in class to explore topics, were the most very 
important as helpful for their thinking. The rest of the seven factors are as follows: (3) “cognitive 
scaffolding” (drawing pictures, teachers’ presenting examples, rules, etc.) (4) “independence” (learner- 
controlled instruction, teacher’s raising student’s curiosity, peer collaboration, challenging task, searching, 
inquiry activity), and (5)” Alternatives” (alternative forms of assessment and peer teaching opportunity), (6) 
“Reflection Process”, and (7) “Reality”. The motivational and affective factors factor included in this study 
were perceived as most helpful. The cognitive scaffolding factor was perceived as least helpful. 

One of the most meaningful findings in this study was the perceived importance of learner- 
centered practices in K-12 education. Most of learning environment elements that were highly regarded as 
helpful were aligned with the characteristics of learner-centered classrooms. Given that the feasibility of 
other’s supportiveness and flexibility factor reflecting young adolescents students ’ motivation and affective 
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factor suggested APA (1997) were emerged as the most helpful factor, this study results support the idea 
that without appropriate support, young students have difficulty engaging in high-level reflective thinking( 
Lin et. al., 1999;Hmelo, & Lin, 2000). In addition, as expected, the fundamental assumption of this study 
that “the characteristics of the learner-centered classrooms including affective scaffolding should be 
perceived as important in young student’s reflective thinking in K-12” was supported. These results have 
implications related to the young adolescents ’ distinctive developmental stages that are characterized on 
“independence” as well as “dependence”. 

More research is needed to investigate what are the features that prompt students ’ reflective 
thinking. Also, further research is needed to examine how both instrumental design principles and 
technology design features can be incorporated to provide a learning environment that supports and 
scaffold teachers’ instruction as well as students ’ learning in K-12. 

In addition, to design a classroom as a technology -enhanced, student-centered learning 
environment, more researches on the implied role and effect of technology for supporting students’ 
reflective thinking as well as on the students ’ perception of the impact of technology will be needed. 
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02SHE: The Instructional-Design Theory For Problem-Based Learning 

In Online Education 

Nari Kim 

Indiana University 

Introduction 

There have been various instructional-design theories related to problem-based learning (PBL) in 
our field so far. However, most of the instructional theories on PBL were basically designed for face-to- 
face classrooms, so it has been difficult to apply them to new learning environments such as the Internet. 
Thus, there is a strong need for instructional theories on PBL which specifically can be applied to this new 
open-ended learning environment because Web-based instruction (WBIJ has been considered a useful 
instructional method for a new paradigm in the information age. Many researchers and practitioners believe 
that the combination of these two key instructional components, PBL and the Internet, can help to realize 
an ideal education for learners. Therefore, the purpose of this instructional-design theory is to present a 
guideline on how to apply PBL in the web-based learning environment for increasing “effectiveness, 
efficiency, and appeal’' (Reigeluth, 1999, p.9) of instruction. Based on the differences between classroom 
and web-based environments, this theory tried to focus on the changes of instructional framework such as 
“type, control, focus, grouping, interactions, and support of learning" (Reigeluth, 1999, p.55), and to 
consider the new relationships among learners, tasks, facilitators, and peers. Also, it notified the different 
type of instructional sequences and the importance of self -regulated learning in web-based instruction. Due 
to the nature of the web, learners can click on any menus and jump into the next step if they want, so the 
sequence of instruction is relatively flexible compared to classroombased instruction controlled by 
teachers. 



Instructional- design theory: 02SHE 

In terms of the instructional sequence applying PBL to the Web-learning environment, this 
instructional-design theory consists of five stages including Opening, Starting, Solving, Helping, and 
Ending (OSSHE). The name of this instructional-design theory, “02SHE," came from the combination of 
first letters of these five stages. The context, values, and the structure of the instruction which were 
considered to design 02SHE are as follows: 

Context of the instructional -design theory 

This theory of instruction, 02SHE, was designed considering the range of context in terms of 
learners, tasks, learning environments, and learning styles. 

Target audiences of this instructional-design theory are adult learners in a non-academic setting. 
Due to the characteristics of this learner type, promoting participation and increasing motivation are 
important through the whole process of the instruction. The number of learners can be flexible based on the 
capacity allowed of the web server. As the task, cognitive skills which can be acquired through authentic 
problems are suitable. This theory is designed for Web-based learning environments as mentioned above. 
However, the types of WBI are various in terms of the computer capability and the internet speed. Thus, 
according to different levels of WBI, several alternative strategies will be suggested as optional methods, 
considering possible situations. Basically, the type of learning supported by this theory is individual self- 
study mentored by facilitators . For this reason, the learning process and schedule of individual learners may 
be different based on the course registering time. However, this theory can be modified for team projects by 
adding collaborative activities. 

Values of the instructional -design theory 

According to Reigeluth, “a major implication of design theory’s goal (or design) orientation and 
emphasis on preferability of methods for attaining its goals is that values play an important role for design 
theories’’(1999, p. 1 2). In 02SHE, the values of the instructional theory are as followings in terms of three 
key elements related to instructional goals and methods: learning environments, tasks, and facilitators. 
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First, learning environments should be learner-centered, and enhance self-regulated learning. Also, 
they should provide various tools for students to share their ideas and feelings and enhance their 
understanding with detailed guidelines for learner activities. Second, tasks in this theory should provide 
customized learning content for learners’ different needs, be related to authentic situations of learners, and 
encourage learners to have divergent thinking and multiple perspectives. In addition, they should be 
designed in order to combine individual activities and group activities, help learners to apply new 
knowledge to their real world, and evaluate the whole process of learner activities in a holistic way. Last, 
facilitators should have attitudes to help learners actively, guide them to construct their own learning, 
provide learners with proper feedback frequently, and help learners to foster higher-order thinking as well 
as to increase their confidence. In this way, they also should let learners feel connected to their facilitators 
in the online environment. 

Structure of the instructional-design theory 

In order to explain the structure of instruction in 02SHE, it is necessary to consider the structure 
of elements of PBL in a classroom environment first. The structure of instruction in PBL in a classroom 
environment can be shown by the following diagram (see Figure 1). It is based on the key elements that 
make up the instruction: a learner, peers, tasks, facilitators, and a learning environment. 

Figure 1. The structure of PBL in a classroom environment 




On the other hand, the structure of the instruction in 02SHE for PBL in the Web-based learning 
environment can be represented as the diagram below (see Figure 2). The instruction in 02SHE has the 
same main elements as those in PBL in the classroom environment including a learner, peers, tasks, 
facilitators, and a learning environment. 

However, there are some differences among the relationships of instructional elements in 02SHE, 
compared to those in the clasroom environment. First, due to the nature of WBI, it may be difficult to 
separate a learning environment from tasks within it. Thus, in the diagram below, a dotted line of tasks 
(PBL) implies an open-ended boundary to the learning environment (Web). Second, the role of facilitators 
may be reduced, compared to instructors who deliver their knowledge in classrooms because a learning 
environment and tasks in 02SHE share facilitators’ roles in terms of providing learners with content. 
Usually, the facilitators in this instruction focus on helping learners to discover instead of providing them 
with correct answers and directed solutions. A thin gray line implies this reduction of facilitators’ roles in 
the diagram below. Last, this instruction is basically for self-study, not for collaborative study, though 
learners may discuss with peers who have their individual problems to be solved through the instruction. 
Thus, the role of peers may be reduced compared to PBL in the classroom. A thin gray line implies also 
this reduction of peers’ roles. 
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Figure 2. The structure of the instruction in 02SHE 




Methods of the instructional-design theory: stages and events of 02SHE 

As instructional methods for PBL in online education, there are five stages (Opeining, Starting, 
Soloving, Helping, and Ending) and ten events suggested in 02SHE (see Figure 3). Following the order of 
the five stages in the direction of arrows is strongly recommended. However, the order of two events within 
each stage can be changed flexibly depending on the purpose of the instruction. The dotted lines of boxes 
below mean open-ended boundaries of the five stages. 
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1. Opening 



■ Share the value of instruction 

■ Present the guideline of instruction 






2. Starting 






■ Help learners prepare 

■ Let learners face problems 


4. Helping 


3. Solving 




1 

| ■ Facilitate learnin 
! ■ Fncouraee learne 


1 

g experience | 

rs to share ideas ! 


■ Provide tools and resources 

■ Promote critical thinking through reflection 


■ O 1 

■ 1 

I j 






5. Ending 






■ Evaluate the process and outcome of learning 

■ Open new knowledge to the real world 





Figure 3. Five stages and ten events of 02SHE 

In the following section, these five stages and ten events in 02SHE will be explained in details. In 
addition, within each of the events, its parts, criteria, and options will be presented as specific guidelines. 
Following the order of the parts may be recommended or not, depending on the situations in each event. 
Also, some of the options may be used together if it would be necessary though they may have different 
strategies. 

1. Opening 

First, there are two events in Opening stage: Share the value of instruction and Present the 
guideline of instruction. 

Share the value of instruction 

This event is for explaining what learners will learn and why theyshould study. Unlike K-12 
students , adult learners do not want to continue studying without strong reasons. Sharing the value of 
instruction in the beginning of the instruction can increase learner motivation and engagement to complete 
instructional programs. The description of the value should be for learners, not for instructors. In addition, 
this step should try to provide learners with a chance to find the meaning of instruction by themselves 
instead of presenting them with the value of instruction. This value of the instruction is different from “the 
values of the instructional theory” (Reigeluth, 1999, p.12), which are mentioned in the previous section. 
There are several parts and their criteria for this event below. Following the order of parts below is 
recommended. 

Clarify instructional goals and objectives: The goals and objectives should identify tasks clearly 
in terms of what learners will learn through the instruction. These should be presented by using simple 
performance-based terms to help learners understand instead of using technical rules about writing 
objectives for instructional designers. 

Show the vision of learning experience: The vision of learning experience should make learners 
realize why they learn and what the instruction is worth. It should inspire learners to complete tasks by 
showing the relevance to their future jobs. 
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Remind learners of their personal aims: Evoking learners’ personal aims is important in terms 
of reminding the learners why they originally wanted to study the course. Thus, it will be helpful to look 
back on their actual reasons for taking the course to increase learner motivation. 

Make a connection between instructional goals and learner’s aims: In some cases, learners 
may have different personal aims from the instructional goals. Thus, it is important to try to reduce the gap 
between instructional goals and learners’ aims. This step should help learners to realize they are studying 
the courses related to their aims in some ways. 

Convince learners of their change though the instruction: If learners completely understand 
why they should go through this instruction, then it is time to help them to have confidence and to believe 
they will be successful. 

Depending on the situations related to the capacity of technical equipment and learners’ intentions 
through this event, optional methods are as follows. 

First of all, if the capacity of learners’ computers and the speed of the internet are low or if 
learners do not have enough time to spend in this event, we can explain the value of the instruction in a text 
form However, if the capacity of learners’ computers and the speed of the internet are low or if learners do 
not think about their personal aims at all, we need to provide previous learners’ success stories as examples, 
let learners consider their personal aims to take the course, and then ask learners to share their experiences 
on the asynchronous forum On the other hand, if the capacity of learners’ computers and the speed of the 
internet are high or if learners want to consider the value of instruction seriously, we can provide short 
video clips about previous learners’ success stories, let learners write a short essay about their future after 
finishing the course, ask learners to discuss the value of instruction from their point of view on the 
asynchronous forum and then provide proper feedback on the discussion. However, although these 
situations have different method strategies, some of these methods can be used together for a certain 
purpose if it would be necessary. 

Present the guideline of instruction 

The guideline of instruction provides generally two kinds of information in this event: how the 
instruction will be heldand how learners can use the web-based environment for their learning 
activities. Especially, in problem-based instruction, it might be easy for learners to lose their focus and 
direction during learning activities because of complex tasks embedded in their problems. Thus, this 
guideline event in the beginning step can help these learners to imagine what they will experience during 
the course. In addition, learners should know unique functions of electronic tools in the website, so they 
should feel comfortable when they find necessary information by using these tools. There are several parts 
and their criteria for this event as follows. However, the order of these parts may be flexible according to 
learner access. 

Present the process of instruction step by step This instructional process should be described 
easily step by step considering novice learners’ points of view. It also should offer enough information to 
cover the whole process of instruction. 

Explain the activities of learners, peers, and facilitators in instruction: Learners should know 
about expected roles of the main participants in the instruction including peers, facilitators, and themselves. 
However, peers’ and facilitators’ roles depend on the purpose of instruction, so their roles may be 
explained in different levels. The effects of dynamic interactions on learning should be understood by all of 
the main subjects who participate in the instruction. Specially, it is important to clarify learners’ activities 
that should be completed through the instructional process because learners’ active participation is the key 
factor to success in problem-based learning. 

Guide how to use the website during the instruction: A Web-based learning environment is still 
new and unfamiliar to many adult learners. Thus, it is critical to present proper information for learners to 
understand new instructional methods that support learning through the web. In addition, it should be 
prerequisite for learners to understand functions of electronic tools and to master technical skills using the 
tools before starting the instruction. This can be the key part that distinguishes the web-based instructional 
theory from classroom-based instructional theories. Learners in Web-based instruction should be as 
familiar with their learning environment as they have been with how to use a blackboard in the classroom 
environment. 

Provide technical support If there might be novice users who have difficulties in the use of 
computers and the internet, they should be helped by facilitators. Thus, proper technical supports should be 
provided instantly through the whole instructional sections. 
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According to the situations in this event, there might be optional methods as follows. 

If learners are not familiar with this type of instruction, we might provide a detailed syllabus 
including specific schedules, grading rules, and assignments. However, if learners need to know more 
about the process of the instruction and their activities, we can provide instructional guidelines on the 
sequence of tasks including tour lessons as an example, FAQ, Site map. Help site, and Practice site for 
tools . Also, if there are enough facilitators to give technical advice related to computers and internet access, 
we can provide Q&A sites and Phone numbers & E-mail addresses of facilitators. 

2. Starting 

In Starting stage, there are two events including Help learners prepare and Let learners face 
problems. 

Help learners prepare 

The most important part of this theory for PBL is to consider when and how we present 
“instructional space,” judging the complexity of the problems (Reigeluth & Moore, 1999, p.66). 
According to Vygotsky (Reigeluth & Moore, 1999), “the zone of proximal development” (p.66) should be 
considered in designing instructional events. This middle stage of instruction should be provided as bridges 
to connect the current level of knowledge to the ideal level of knowledge. Especially, in the case of 
problem-based learning, this stage should be presented as the “instructional space” in which learners should 
prepare necessary knowledge, skills, and attitudes before entering complex and ill-structured problems. 
Several parts and their criteria for this event are presented below. Following the order of parts below is 
recommended. 

Test learners’ prior knowledge: Usually, adult learners have more different experiences and 
knowledge compared to K-12 students when they start a new instruction. Thus, it is important to check out 
their prior knowledge and to help them prepare by providing proper instructional treatments. It may be easy 
to guess learners’ prior knowledge would be the same, but it should not be assumed without pre-tests. To 
get desirable results with the prior knowledge tests, task analysis should be conducted carefully in the 
instructional design process. Multiple choice or short essay questions can be used as the pre-tests. 

Provide warm-up lessons to reduce the knowledge gap before starting PBL: Based on the test 
results on learners’ prior knowledge, different warming-up lessons should be provided as “instructional 
space” for individual learners. This step should try to provide customized content for reducing learners’ 
different knowledge gap. However, these types of customized warm-up lessons, which satisfy different 
levels of learners, can increase the cost of developing instructional programs. Complex problem-based 
learning types are not appropriate as warm-up lessons because they can cause learners heavy cognitive 
loads in the beginning of the instruction. The length of warm-up lessons can be flexible depending on 
learners and tasks, but it should be shorter than the main instruction. 

Help learners have confidence to challenge instruction: Some learners may be disappointed 
with the results of their prior knowledge tests. In this case, facilitators should help learners to get 
confidence again and to continue warm-up lessons. After finishing the warm-up lesson step, facilitators 
need to remind learners they are ready for moving onto the next level and also encourage learners to 
challenge themselves. 

According to the different levels of learner prior knowledge, there might be optional methods as 

follows. 

If learners show lack of several types of prior knowledge, we can provide separate prerequisite 
lesson modules from this instruction in order to get the proper level of prior knowledge on the task, and 
give learners a chance to restart this instruction after finishing their preiequisite lesson modules. However, 
if learners have enough prior knowledge, but they have difficulty remembering it, we need to present a 
short explanation webpage to stimulate and recall their existing knowledge. In addition, if learners have 
enough prior knowledge and they remember it very well, it will be necessary to let learners start problem- 
based learning, skipping warm-up lesson steps. 

Let learners face problems 

In this event, learners will face authentic problems as “heuristic tasks” (van Merrienboer, 1997) 
related to their real contexts which should be solved through the instruction. Using a real situation problem 
can be helpful to gain attention, to make the problem tasks familiar, and to provide strong motivation to 
challenge. If it is necessary, several learners in a team can choose their team’s problem. However, this 
instruction will recommend that individual learners select their own problems related to the task because 
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each learner will have their own different situations in their lives. The following several parts and their 
criteria can be used for this event. Keeping the order of parts is recommended. 

Ask learners to think about possible problems related to the task in their contexts: Learners 
should know that the task is based on their instructional goals and objectives. Also they should be asked to 
consider specific real situations that can provide problems related to the task. 

Let learners have authentic problems which should be solved through the instruction: The 
authentic problems should be heuristic tasks which consist of chunks. This step should be applied in 
different ways based on learners’ prior knowledge and their contexts as mentioned in optional methods 
below. 

Help learners to discover what the problem is and what they shoul d do with it: It is important 
for learners to discover what they will solve embedded in the situation. This step should help the learners to 
see their problems from their own points of view. If it is necessary, facilitators can help them to find the 
most important issues in the problems. 

Let learners realize necessary resources for new concepts, procedures, and principles which 
will be mastered to solve the problem: Learners should spend proper amount of time for figuring out 
which kinds of knowledge will be necessary to complete their task through solving problems. In order to 
master this new knowledge such as concepts, procedures, and principles, learners also should recognize 
which kinds of resources will be necessary. 

Depending on the different levels of learner prior knowledge, there may be optional methods as 

follows. 

If learners feel difficulties in understanding the purpose of problem-based learning, we can present 
information on how to use problems through the instruction again though this kind of information already 
should be presented in the guideline page of the instruction. Also, it will be necessary to let students reflect 
their personal context related to the task again, and ask learners to select one of the problems suitable for 
their context . However, if learners feel difficulties in finding their authentic problems directly related to the 
task, we may present authentic situation problems prepared for the instruction. On the other hand, if 
learners have their actual problems directly related to the task, we might let students write down their 
personal context related to the instruction, provide the guideline about how to choose proper problems in 
learners' context to be solved through the instruction in a boundary of the task, and ask learners to find their 
own real problems in their contexts. 

3. Solving 

In Solving stage of 02SHE, there are two events called Provide tools and resources and Promote 
critical thinking through reflection. 

Provide tools and resources 

By using given tools and resources, learners can search for necessary information, remember and 
understand new knowledge (e.g., concepts, procedures, and procedures), and apply their understanding to 
the problem situations. The balance of given tools and resources can determine the level of discovery 
learning in the instruction. This event might consist of activities for individual or collaborative learners if it 
is necessary. The following several parts and their criteria can be used for this event. These parts are 
recommended to be applied in the order below. 

Supply appropriate tools: Learners should be provided with proper tools that can scaffold their 
abilities to complete the tasks through the whole process of solving the problem. As tools for finding 
resources, all kinds of electronic computer tools can be used such as web-searching engines and word 
processing programs. Also, these tools can include cognitive tools that help to visualize, organize, and 
supplement thinking skills (Jonassen, 1999, p.226). 

Demonstrate an example of how to search for information by using tools: As cognitive 
apprenticeships, the expert’s performance of searching for necessary information with given tools should 
be presented to novice learners. This may be a type of procedure that demonstrates technical skills even 
though this lesson is for cognitive skills. 

Let learners discover how to find necessary resources: If the instruction emphasizes discovery 
learning, relatively more tools can be supplied than resources. Resources usually are meant as new 
information and knowledge and they may be various reading materials, useful websites, real cases related 
to the problems, and drill and practice. 
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Help learners to learn new knowledge through the found resources : In this step, learners 
should remember and understand new concepts, procedures, and principles embedded in the resources 
found by themselves. 

Let learners apply acquired knowledge to the process of solving their problems : Learners 
should try various strategies based on the new knowledge to their problem situations. 

Encourage learners to repeat the cycle of this process until they solve the problem 
completely: Through the cycle of this process, learners should reduce their anxiety and challenge higher 
level tasks. 

Depending on the situations to provide tools and resources, optional methods are as follows. 

If learners select one of problems suitable for their context, it will be a good way to provide the 
URL of websites as resources for solving their problems , and to present guidelines on how to select proper 
information among the bunch of resources . On the other hand, if learners find their own problems in the 
actual context to be solved through the instruction in a boundary of the task, we can focus on providing 
more useful tools in finding resources suitable for their problems rather than resources, and guide learners 
not to loose their focus in searching for information. 

Promote critical thinking through reflection 

Critical thinking can be an important component to improve deep understand! ng as higher-order 
thinking (Garrison, Anderson & Archer, in press). Through the critical thinking process, learners may 
figure out their own approaches on how to solve problems. For example, they can find how to make 
hypotheses for solving the problems as a first step and how to use resources to support the hypotheses. In 
order to promote this critical thinking, asking individual reflection can be a good strategy. This reflection 
might be considered for evaluation event as one of the ongoing assessment strategies if it is necessary. The 
possible parts and criteria for this event are as follows. The parts of this event might be applied in the order 
below. 

Provide a guideline on how to think critically: Most learners feel unfamiliar with the critical 
thinking process though they unconsciously think in that way during their lives. Thus, it may be necessary 
to provide learners with a small guideline to explain how to think critically. The guideline should include 
the four phases considered in the process of critical thinking: “trigger, exploration, integration, and 
resolution” (Garrison et al, in press). Thus, learners should realize how to identify or recognize the problem 
in Trigger phase, how to expend their initial ideas through brain storming in Exploration phase, how to 
construct meaning from the generated ideas generated in Integration phase, and finally how to implement 
the proposed solution in Resolution phase. 

Present examples on how to solve problems through critical thinking: It could be a good 
method to provide proper examples on how to solve problems . The examples should be authentic cases 
related to learners’ context and problems. 

Ask learners to review their thinking process through reflection : This step may be applied 
differently according to the schedule of the course. Learners might need to write daily diaries on their 
reflection every day or submit short reflection papers in certain points of the instructional process . 

Also, depending on the purposes of the instruction, there may be optional methods as follows. 

If individual reflection is important for the instruction, it might be a good way to provide 
individual spaces in websites which learners can write and keep their reflection papers . However, if 
collaborative reflection is essential, it would be better for learners to share their reflection papers with other 
peers. On the other hand, if there are enough numbers of facilitators, they can provide appropriate feedback 
on individual learners’ reflection about their critical thinking processes. 

4. Helping 

There are two events in Helping stage: Facilitate learning experience and Encourage students to 
share ideas. Although Helping stage is explained here as the fourth stage in 02SHE, this stage should be 
used flexibly for the other four stages whenever it is necessary. 

Facilitate learning experience 

Through proper feedback and guides, facilitators should try to expand learning experiences in 
three ways: “social presence, cognitive presence, teaching presence” (Garrison et al, in press). Especially, 
the role of facilitators is important for learners to continue web-based instruction. The possible parts and 
criteria for this event are as followings. The parts of this event might be applied together at the same time. 
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Develop social relationships between learners and facilitators : Frequent and regular feedback 
is very important to make a trust between learners and facilitators. Facilitators should use the proper social 
presence cues such as personal feelings and emotions. 

Help learners to review their cognitive process of approaching problems : This is related to the 
critical thinking skills of learners, but it is important to notice that individual learners’ cognitive process 
can be improved by facilitators’ guidance. Two kinds of strategies may be used for this step. First, 
facilitators should Guide learners to sharpen the focus on the issues. They should push the individual 
thinking forward, clarifying goals and expectations. “Identifying direction, sorting ideas for relevance, and 
focusing on key points’’ can be asked for in this step. Second, facilitators should aid learners to think of the 
problems in a proper boundary. Learners should see what they may have over looked and examine their 
own assumptions. As the three specific strategies in this step, “full -spectrum questioning, making 
connections, and honoring multiple perspectives’’ can be used (Garrison et al, in press). 

Assist learners to follow the schedule of the course: Through the whole instruction, study 
processes of learners should be managed by facilitators, using various electronic tools such as log-in data 
and tracking functions. If some learners do not catch up the recommended process of the instruction, 
facilitator should check out the reasons and provide proper treatments for them to finish the course. It is 
essential that learners feel facilitators’ care during participating in the instruction. 

Aid technical problems related to computers and the internet If there might be learners who 
have difficulties in Web-based instmction, they should be helped by facilitators. Facilitators should provide 
appropriate technical supports immediately through the whole instructional sections. 

According to the number of facilitators, alternative options can be used as follows. 

If facilitators are enough for the numbers of learners, we may match a primary facilitator to each 
individual learner. However, if facilitators are not enough for the numbers of learners, it will be better to 
provide team facilitation and a more specific guideline webpage such as Q&A and to promote learners to 
help other peers instead of facilitators . 

Encourage learners to share ideas 

This event is for collaboration and discussion. Through this process, learners should be 
encouraged to discuss their ideas on problems deeply and widely or combine their efforts together for 
finding the best solution. Through the process of sharing ideas with peers, it is important to look for 
potential solutions from different perspectives. However, these collaborations and discussions depend on 
the types of the instruction. If learners need to continue the instruction as self-study, this collaboration may 
not be used for this step. Nevertheless, it will be important to share their opinions and questions obtained in 
the process of new learning experience. Learners can see their current problems from a different 
perspective through sharing new ideas with peers. There are several parts and criteria that can be 
considered as follows. The order of these parts may be flexible. 

Provide proper discussion places to share ideas: Deciding the type of discussion places depends 
on the types of instruction. If learners study collaboratively, team discussion places should be provided for 
sharing ideas as well as team works (e.g., uploading files in synchronous forum web pages). Also, proper 
discussion solution should be selected carefully considering the number of students and facilitators. 

Help to generate rapport among learners through the discussion: Rapport is very essential for 
peers to open their minds and share their individual ideas without hesitating. Facilitators should help to set 
up rapport in the initial stage of the discussion especially for adult learners. 

Cheer learners to participate in discussion actively During the discussion, facilitators should 
promote learners to participate in the discussion and try to prevent a few learners from dominating the flow 
of discussion. 

According to the capacity of computers and the internet, there may be different situations and their 
optional methods as follows. 

If the capacity of learners’ computers and the speed of the internet are low or if learners do not 
need instant feedback from their peers, we can provide asynchronous forum web pages and promote using 
personal emails . However, if the capacity of learners’ computers and the speed of the internet are high or if 
learners need instant feedback from their peers, it can be better to provide asynchronous forum web pages 
and to provide customized synchronous forum web pages. 

5. Ending 
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Last, Ending stage has Evaluate the process and outcome of learning and Open new knowledge to 
the real world as two specific events. 

Evaluate the process and outcome of learning 

This event is for assessing what learners learn through the instruction. Both of the process and 
outcome of learning should be considered during/after the instruction. The possible parts and criteria for 
this event are as follows. The order of these parts is recommended to follow. 

Evaluate the process of learning during the instruction: Ongoing evaluation should be 
conducted often in the learning process. To improve learning activities, facilitators should let learners know 
their evaluation results and comments during the instruction. For this reason, this step can be related to 
facilitators’ feedback in terms of providing guidelines regarding the next learning step. 

Evaluate the outcomes of learning after finishing the instruction: For this step, performance- 
based evaluation should be emphasized in terms of helping learners to retain their knowledge in real 
situations. Like the learning process assessment step, providing proper feedback on learning outcomes is 
still very important even though learners already would finish the whole instruction. Thus, it would be good 
strategies to offer final comments including strengths and improvements on learners’ activities and 
products with their grades. 

According to the purpose of the instruction, there may be optional methods. 

If the instruction is self-study, it would be possible to count students’ login data, posted messages, 
and schedule tracking for individual learning processes, and also use self-evaluation report for their 
outcomes with facilitators’ evaluation. Or if the instruction is collaborative study, it would be better to 
include peer-evaluation with self-evaluation. 

Open new knowledge to the real world 

Learners should be encouraged to apply new knowledge to their real worldafter finishing the 
course. Possible parts and criteria for this event are as follows. The parts of this event might be applied in 
the order below. 

Summarize new knowledge and skills : It can help students to recall their new knowledge 
acquired during the course. After finishing the evaluation section, learners can get their own insights to see 
their problems from different perspectives. Thus, providing the big picture of the instruction and the flow 
of the key lesson points would be very effective to retain the new knowledge and skills to their 
performance. 

Ask learners to apply acquired new knowledge to their real world This step should help 
students to connect their new knowledge to their real lives over the boundaries of the instruction. For this, 
learners might be encouraged to find various examples related to the new knowledge in their real lives. 
Post-instructional applications may be useful to check out learners’ retention after finishing the courses. 

According to the situations related to the common content, there may be optional methods as 

follows. 

If there is common content among the process of solving their problems, we may provide short websites to 
summarize the new concepts, procedures, and principles. However, if it is not easy to find common content 
among the process of solving their problems, we may ask them to summarize acquired new knowledge by 
themselves through reflection papers. 



Summary 

As a possible guideline on how to apply problem-based learning in the web-based learning 
environment, we have considered five stages, ten events, and their parts and criteria of 02SHE. Also, we 
have thought about several alternative methods depending on the situations that are related to the outcomes 
of instruction such as “effectiveness, efficiency, and appeal’’ (Reigeluth, 1999). This theory, 02SHE, might 
be one of the instructional-design theories which accept actively changes of instructional framework and 
new relationships among learners, tasks, facilitators, learning environments, and peers for PBL in online 
education. 



References 

Collison, G. Elbaum, B., Haavind, S., & Tinker, R. (2000). Facilitating online learning: Effective strategies 
for moderators. Madison, WI: Atwood Publishing. 



357 




Garrison, D. R., Anderson, T., & Archer, W. (in press). Critical thinking and computer conference: A 
model and tool to assess cognitive presence. American Journal of Distance Education. 

Jonassen, D. (1999). Designing constructivist learning environment. In C. M. Reigeluth (Ed), Instructional 
Design Theories & Models: A New Paradigm of Instructional Theory (2 nd ed., pp. 215-239). 
Mahwah, NJ: Lawrence Erlbaum Associates. 

Reigeluth, C. M. (1999). What is instructional-design theory and how is it changing. In C. M. Reigeluth 

(Ed), Instructional Design Theories & Models: A New Paradigm of Instructional Theory (2 nd ed., 
pp. 5-29). Mahwah, NJ: Lawrence Erlbaum Associates. 

Reigeluth, C. M. & Moore, Julie. (1999). Cognitive education and the cognitive domain. In C. M. Reigeluth 
(Ed), Instructional Design Theories & Models: A New Paradigm of Instructional Theory (2 nd ed., 
pp. 51-68). Mahwah, NJ: Lawrence Erlbaum Associates. 

Van Merrienboer, J. J.G. (1997). Training Complex Cognitive Skills. Englewood Cliffs, NJ: Educational 
Technology Publications. 



358 




Determinants for Failure and Success of Innovation Projects: 

The Road to Sustainable Educational Innovation 

P.A. Kirschner 
M. Hendriks 
F. Paas 
I. Wopereis 

Open University of the Netherlands 

B. Cordewener 
SURF Foundation 
The Netherlands 

Abstract 

Robert Burns wrote: “ The best laid schemes of Mice and Men oft go awry ”. This could be 
considered the motto of most educational innovation. The question that arises is not so much why some 
innovations fail (although this is very important question), but rather why other innovations succeed? This 
study investigated the success factors of large-scale educational innovation projects in Dutch higher 
education. The research team attempted to identify success factors that might be relevant to educational 
innovation projects. The research design was largely qualitative, with a guided interview as the primary 
means of data collection, followed by data analysis and a correlation of findings with the success factors 
identified in the literature review. In order to pursue the research goal, a literature review of success 
factors was first conducted to identify existing knowledge in this area, followed by a detailed study of the 
educational innovation projects that have been funded by SURF Education. To obtain a list of potential 
success factors, existing project documentation and evaluations were reviewed and the project chairs and 
other important players were interviewed. Reports and evaluations by the projects themselves were 
reviewed to extract commonalities and differences in the factors that the projects felt were influential in 
their success of educational innovation. 

In the next phase of the project experts in the field of project management, project chairs of 
successful projects and evaluators/raters of projects will be asked to pinpoint factors of importance that 
were facilitative or detrimental to the outcome of their projects and implementation of the innovations. 

After completing the interviews all potential success factors will be recorded and clustered using an affinity 
technique. The clusters will then be labeled and clustered, creating a hierarchy of potential success factors. 
The project chairs will finally be asked to select the five most important success factors out of the 
hierarchy, and to rank their importance. This technique - the Experts' Concept Mapping Method- is based 
upon Trochim ’s concept ma pping approach ( 1 989a, 1 989b ) and was developed and perfected by Stoyanov 
and Kirschner ( 2004 ). 

Finally, the results will lead to a number of instruments as well as a functioned procedure for 
tendering, selecting and monitoring innovative educational projects. The identification of success factors 
for educational innovation projects and measuring performance of projects based upon these factors are 
important as they can aid the development and implementation of innovation projects by explicating and 
making visible (and thus manageable) those success and failure factors relating to educational innovation 
projects in higher education. 

Determinants for Failure and Success of Innovation Projects: The Road to 
Sustainable Educational Innovation 

The Dutch Government has invested heavily in stimulating better and more creative use of 
information and communication technologies (ICT) in all forms of education. The ultimate goal of this 
investment is to ensure that students and teachers are equipped with the skills and knowledge required for 
success in the new knowledge-based economy. All stakeholders (i.e., government, industry, educational 
institutions, society in general) have placed high priority on achieving this goal. However, these highly 
funded projects have often resulted in either short-lived or local successes or outright failures (see De Bie, 
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2003; Teasly, 1996). As a result, the role of ICT is developing less quickly in higher education institutions 
than was previously expected. 

In order to steer these developments in the right direction, SURF, a government-funded national 
organization in which all higher education institutes in the Netherlands participate to increase the pace of 
educational innovation, first set up the SURF Educatie<F> (SURF Education) program for educational 
innovation projects and has followed this up with the SURF ICT and Education Platform. The goal of this 
platform is the systematic stimulation of the application of ICT to innovation in higher education via: (1) 
professionalization of personnel, (2) effective deployment of resources, (2) monitoring the durability of 
(digital) educational material, (3) facilitating measurable improvements in education as a result of the 
deployment of ICT, (4) fostering a systematic approach to innovation and the development and 
dissemination of knowledge, and (5) promoting the use of standards. Finally, SURF’s educational 
innovation projects are intended to be a source of inspiration for the introduction of ICT-based innovation 
in education and are aimed at better and sustainable results. It’s educational innovation projects often cover 
one or more of the following key issues: 

• Competence/Portfolio 

• Collaborative learning 

• Interactive teaching materials 

• Learning content management systems (LCMSj/Communities 

• New media 

This is all very noble, but after having funded projects for five years, SURF asked itself three 
important questions with respect to educational innovation, namely: Why are some innovations more 
successful than others? Why do some innovations fail, while others succeed? and How can an innovation 
be sustained once the grant funding has ended? The present project, funded by the SURF ICT and 
Education Platform, tries to answer these questions by identifying the determinants for success and failure 
of large-scale educational innovation projects in Dutch higher education, in particular of those funded by 
the SURF ICT and Education Platform. The identification of success and failure factors for educational 
innovation projects and measuring the chances and performance of projects based upon these factors are 
considered important to SURF Education as they can aid the development and implementation of 
innovation projects by explicating and making visible those success and failure factors relating to 
educational innovation projects in higher education. This is necessary to make these factors manageable 
and to enable future projects to achieve better and more sustainable results. 

The research project consists of the following phases: 

• A literature study to determine the benchmarks for success and failure of educational innovation 
projects. 

• Analyses of the SURF projects started in 1999, 2000, and 2001 on the basis of the benchmarks from 
the literature study and the identification of new benchmarks from these projects. 

• Interviews with experts from SURF Education, project managers of the SURF Education projects and 
experienced (commercial and governmental) project managers and advisors to determine the subjective 
dimension of success and failure. 

• Development of a number of tools for evaluating project plans and progress as well as a new procedure 
for tendering for funds. 

• Dissemination of the results by writing articles for professional and scientific journals and the 
presentation of the results at national and international conferences and workshops. 

This contribution presents the results of the first phase of the research project; the literature review 
of success factors of educational innovation projects to identify the existing knowledge in this area. 

A first step in creating a meaningful report is finding an answer to the questions of how ‘success’ 
can be operationally defined and how the dimension ‘success of educational innovation’ can be tapped. A 
problem here is that there are two types of ‘success’, namely success of the project and success of the 
innovation. These two ‘successes’ are completely different. Neither is a requirement nor a guarantee for the 
other. In general, success can be seen as the accomplishment of goals and objectives necessary to achieve a 
particular task. SURF ICT and Education Platform projects are considered successful based on the extent to 
which they stimulated and facilitated new and better use of ICT in education. SURF is particularly 



360 




interested in the sustainability of the technological innovations, in other words, how they can ensure that 
the innovations, both technical and pedagogical, make the shift from an externally funded initiative to a 
sustained ‘standard operating procedure’. The rationale for sustaining successful ICT -based educational 
innovations is to preserve what has been valued and built for continued use. 

Inherent to the concept of success factors is a notion that if success factors are implemented / 
heeded in the project, the educational innovation or project will perform better and be successful. Success 
factors can thus be looked upon as individual independent variables influencing the dependent variable 
‘success’. Analogously for failure factors, there is a notion that if failure factors are avoided in the project 
or implementation, that the innovation will have a better chance of being achieved. Performance 
measurements for success or failure can be classified as objective and subjective. The objective measures 
are based on measurement of past performance or output while subjective measures let individuals with an 
intimate knowledge of educational innovation be the judges of what is successful. It should be recognized 
that success of an innovation at the project level does not automatically generalize to success at the 
institutional or national level, but needs careful considerations of scalability, generalizability, temporal 
flexibility, and financial sustainability. To be able to determine which of the identified success factors for 
educational innovation projects are most relevant in different educational contexts, a contextual framework 
to position educational innovation and related success factors needs to be developed. The basic idea is that 
if we know why some projects fail or have only short-lived successes and we can avoid making these 
mistakes, it is possible to make sure that new projects will succeed with sustainable results. However, it 
should be noted that ‘non-failure’ is not considered the same as success. Success/failure should not be 
considered as a binary classification, but should be viewed along a continuum. 

Although, SURF sets clear goals for applicants to address sustainability in their project plan, and 
generally, clear implementation plans are available, clear plans for sustaining the innovation are lacking. 
Sustainability seems to be more an afterthought rather than a planned strategy for maintaining change. 
Kenny and Meadowcroft (1999) suggest that forward thinking and vision are paramount in successfully 
planning sustainable developments. The observations presented in the literature study are meant to 
contribute to the understanding of why and how technological innovations in education are adopted and 
diffused. Together with the outcomes of the other phases of this project, the report is aimed at supporting 
SURF to steer the innovation projects towards success and sustainable technological innovations and to 
create mechanisms that empower all stakeholders to sustain innovative developments. 

For this literature study, we looked for books, accessed ERIC®, PsycINFO®, and used Google® 
to search the World Wide Web using the following search terms: success, success factors, failure factors, 
success determinants, innovative projects, innovation, return -on-investment, project, innovative factor, 
educational innovation, critical success factor, criteria, guidelines, project management, sustainability, 
success conditions, innovation processes, PT3, standards, Europe, strategy. We located over eighty articles 
specifically addressing the topic of interest. Reading these and following up cross-references we 
established a knowledge base of over thirty books, articles, websites, and papers that addressed different 
aspects of why educational innovations sometimes fail and sometimes succeed. Also, it has resulted in the 
following structure for the rest of the review. 

This report has the following structure. Since innovation can be considered a design and 
implementation exercise composed of a number of distinct phases, the different phases that can be 
distinguished in innovative projects are first described. Second, the business or corporate view on the 
success of innovative projects is described. In business there is a long tradition of using (technological) 
innovation as a major instrument to compete, survive and grow. In comparison, the field of education has 
only recently started to worry about competition and surviving. Third, the experiences from educational 
projects are considered to determine the success and failure factors of innovation projects. Fourth, the 
factors that determine the sustainability of innovations are discussed. Finally, the different experiences are 
synthesized in an integrative model that can be used by SURF to assess project proposals to SURF 
Education regarding their potential to accomp lish sustained innovations, to recognize project warning 
signs, and to increase the chance of success of the diffusion and permanence of the innovations. 

Literature review 

Project phasing 

Innovating and changing an organization is becoming more and more complex because 
organizations and the relationships between organizations are becoming more complex. Factors influencing 
this complexity are level and type of technology, environmental influences, size and structure of the 
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organization (e.g., tendency towards fusions), interdependence between organizations (e.g., tendency 
towards IT production), willingness to change (e.g., overcoming human and organizational inertia), lack of 
support from the management, time and money constraints, and so on. Innovating (or changing) the 
structure of an organization often comes up against a wall of resistance. Not surprisingly, the human factor 
is often considered the most influential factor on the chance of success. Innovating an organization or 
structure places a heavy burden on the organization and the employees and therefore it is necessary to have 
insight in the complex matter of the phases of organizational changes, the way people deal with innovation 
processes, the methods and strategies for change. 

Katz and Kath ( 1 978), for example, argue that planning to initiate an innovation can be done in at 
least three phases: a diagnostic phase in which aspects like how the organization is structured and 
organized need to be addressed, a goal-setting phase where the goals need to be set and the design needs to 
be specified and an innovative-process phase consisting of determining which strategies, roles, methods 
and interventions can best be used to realize the desired situation. Kor and Wijnen (2001) specify this 
concept of phasing a bit further and according to them a project can be divided into six phases: initiation, 
definition, design, preparation, realization and maintenance. Another more condensed and very useful 
model in which the maintenance (or sustainability) of the innovation is explicitly mentioned, is presented 
by Alexander and and McKenzie (1998) and Fullan (1991). In their view on the phasing of organizational 
innovation and change, they distinguish between three main types of phases: an initiation phase in which 
planning and evaluation takes place, an implementation phase in which the development, implementation 
and evaluation of the project takes place, and finally a maintenance or institutionalization phase regarding 
the sustainability of the project. We have opanded this model as shown in Figure 1. 
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Figure 1 Phasing 

Corporate / business point of view 

Within a corporate or business point of view, research on successful projects has been done by 
different studies. Some striking results come from the Standish Group 2000 Study (Johnson, Boucher, 
Connors, & Robinson, 2001): Only 28 % of the IT projects investigated were successful. Since not all un- 
successful projects can be rated as failed the Standish Group 2000 study categorises projects into three 
types: 

• Successful (28%): project is completed on time and on budget with all features and functions originally 
specified; 
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• Challenged (49%): project is completed and operational, but over budget, late, and with fewer features 
and functions than initially specified; 

• Failed (23%): project is cancelled before completion, or never implemented. 

The Standish Group 2000 study and numerous other studies conclude that the human factor is 
often considered as the most influential factor on the chances of failure or success (e.g. Johnson et al., 

2001; Kor & Wijnen, 2001; Schein, 1995; Storm & Jansen, 2004; Turner, 1999) 

Educational point of view 

In addition to the above mentioned studies which were conducted from a corporate or business 
point of view, there has been a good deal of research within educational settings into the determinants of 
success or failure of innovative projects. A two-year national study to evaluate the contribution of 
information and communication technology projects to student learning in higher education was conducted 
in Australia by Alexander and McKenzie ( 1998). The study reviewed over 100 projects, which received 
teaching developments grants and made significant use of a range of information and communication 
technologies to develop student learning materials. After a detailed literature review, a questionnaire was 
developed and sent to the project leaders of 173 projects across Australia and finally these questionnaires 
were analyzed. A striking outcome of the questionnaire was the large discrepancy between the intended 
outcomes of the projects and the actual outcomes reported. While 87% of the projects’ leaders noted 
“improved quality of learning” to be an intended outcome of the project, only 30% reported this as an 
actual outcome . Although this discrepancy can be due to failure of the project and/or failure of the project 
to measure this, it seems justified to state that the majority of the projects have not been successful in 
achieving their intended outcome. Furthermore, the study has shown that technology on itself, does not 
make a difference improving learning outcomes or assures successful educational innovations. 

Diffusion is the process by which an innovation is communicated through certain channels over 
time among members of a social system. The diffusion of innovations theory (Rogers, 1995) helps to 
explain and analyze how an idea or proposal for an innovative project is communicated and accepted by 
others. Therefore, this theory affects the initiation phase and needs to be taken into account when initiating 
an innovative project. According to Rogers (1995, p. 6), “diffusion is a kind of social change, defined as the 
process by which alteration occurs in the structure and function of a social system.” Rogers also defined 
adoption as being a decision to make full use of an innovation and rejection as a decision not to use an 
innovation. It can be concluded that innovations that are perceived by potential adopters as having greater 
relative advantage, compatibility, testability, observability and less complexity will be adopted more easily 
than other innovations. 

Educational innovations, such as implementing a new pedagogic method such as problem-based 
learning, are sometimes the result of an intuitive and hasty decision to change. And without a proper 
investigation of aspects as problems the innovation is supposed to solve, needs of the future users, 
willingness of different participants to cooperate and success and fail factors of this particular project, the 
success of the innovation is at stake. According to De Bie (2003), in order to be successful, projects need to 
take nine success factors into account when initiating an innovation: vision, strategy of development, 
acknowledgement of problems, project plan, project management, resources, role of the “outside world”, 
support from the rest of the institute and competent management. 

Apart from these process aspects, educational innovation has another important aspect, namely the 
content. Both process and content need to be taken into account in order for the project to be successful. 

The content aspect of the educational innovations not only concerns the actual content of the innovation as 
seen in the goals of the project plan, but, according to Vinkenburg (2003), also includes the knowledge, 
skills and attitudes of the people involved in the innovation -as far as this concerns new and to be 
innovated knowledge, skills and attitudes in order to let the innovation succeed. Process aspects are highly 
dependent on the competence of the management, a human factor. Numerous projects fail due to 
incompetent leadership or management (e.g., Alexander & McKenzie, 1998; De Bie, 2003; De Koning & 
Florijn, 1995; Holmes, 2001; Johnson et al., 2001; Kor & Wijnen, 2001; Mathias & Rutherford, 1983; 
Rutherford, 1992; Storm & Jansen, 2004), lack of support from the head of department, dean or other 
person in authority (e.g., Alexander & McKenzie, 1998; Hannan, English, & Silver, 1999; Johnson et al., 
2001; Light, 1998) or lack of support from the rest of the organisation and peers (e.g,. Hannan, English & 
Silver, 1999; Light, 1998; Zhao, Pugh, Sheldon, & Byers, 2002). 
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Sustainability 

As mentioned previously, an innovation process takes place in three stages: initiation, 
implementation and the institutionalization. Although the last phase is often forgotten or neglected (due to 
lack of money, time, etcetera), it is very important. In order to make the time, money and effort worthwhile 
it is important to carefully plan this last phase. The implementation phase may continue for a period of 
time, but eventually there should be a point at which the new idea becomes institutionalized and regularized 
as a part of the ongoing operations. It is now no longer an innovation process, but rather a normal process. 
And whether or not the innovation becomes a durable part of the organization depends on the commitment 
and action of the participants as well as on other factors. 

Schein (1995) found that human change, whether at an individual or group level, is a 
psychological process that involves painful unlearning an relearning while individuals attempted to 
restructure their thoughts, perceptions, feelings and attitudes. People need to un-freeze, change and re- 
freeze. Un-freezing refers to the removal of the restraining or balancing loops that are often associated with 
group norms and leads to cognitive dissonance or conflict that can be very disorienting to group members 
as they begin to change. When dealing with such a disorientation or disequilibrium, the group members 
need to change or reframe their thought process, their ideas and representations of what is “normal’' and 
interpret new concepts more broadly than before. This is called re-freezing. The key to effective change (or 
innovation) is to carefully dose the amo unt of change, and therefore of the perceived threat, produced by 
the disconfirming information to allow the group members to feel safe to un-freeze, change and re -freeze 
their ideas and concepts. But just to receive disconfirming information is not enough to change. To become 
motivated to change, you must accept the information and connect it to something you care about. The 
information must be valid and relevant. Furthermore, O’Hara, Watson, and Kavan (1999) state that the 
more an innovation plans to change, the greater the influence from the environmental will become, the 
greater the risks will be and the chances of failure will increase. 

Light (1998) noticed that many innovations are deserted not at the initiation or implementation 
phase but in the institutionalization phase. He further argued that four main factors influence the degree to 
which innovations are sustained through this institutionalization phase: external environment, internal 
structure, leadership and internal management. Light suggests with this model that when changing the 
structures and culture of an organization, you need to turn the traditional organization into a “learning 
organization’’, that is an organization that adapts to innovations and restructures itself to accommodate 
change. 



Synthesis 

In this paragraph we synthesize the different (educational and business) perspectives and present 
an integrative model that might be used by SURF as a guide in establishing policies and procedures, in 
formulating new guidelines for project proposals, in the assessment of SURF ICT and Education Platform 
innovation project proposals, in conducting a project’s health check to identify warning signs for failure, 
and in promoting the diffusion and sustainability of the projects’ innovations. The model presented below 
shows the factors that are considered imperative to accomplishing a successful project, categorized by 
project stage. These factors are formulated in such a way that they can easily be converted into a checklist 
format that can be used by SURF in the assessment of the project (proposals and reports) and by applicants 
in the writing of project proposals. It should be noted that although these variables are identified as major 
contributors to project success, they will never guarantee success alone. 

Initiation : 

In the initiation phase the present situation needs to be assessed in terms of goals, problems, and 
discrepancies. Then, ideas for the new situation need to be planned taking into consideration the changes 
envisioned and the route from the present to the new situation. Next, the consequences of the effects of the 
innovation on other aspects of the organization need to be evaluated. The following factors are considered 
important in this phase: 

• There should be clear project objectives 

• The general mission of the project should be clearly defined 

• The project’s scope should be adjusted in such a way that the level of changes needed on the route 
from the present to the new situation can be clearly envisioned 

• (Simplified) Return -On-Investment should be taken into account 
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• Stakeholders should be identified 

• Formal feedback channels must be created 

• There should be an experienced project manager 

• The project manager should not be the organizational manager 

• The project manager should be given responsibility and authority 

• The project team members should be competent 

• Care must be taken of clear responsibility and accountability of team members 

• The project manager and team members should be able to explain their efforts and results in ways that 
the larger organization can understand 

• Project manager (team) should be prepared to re -plan 

• The project manager and team members should listen to resistors of innovation because they are often 
aware of unintended consequences of the innovation 

• There should be commitment from executive management and peers 

• There should be support from executive management and peers 

• Realistic expectations should be created 

Implementation: 

In the implementation phase the innovation is developed, implemented, and evaluated. The 
following factors are considered important in this phase: 

• User involvement should be ensured 

• User expectations about the innovation should be managed 

• Stakeholders should be engaged 

• Initiator of the change should be trusted and respected by the prospected users 

• Adequate communication channels should be created 

• Focus should be on adoption rate of approximately 25% of the system members 

• Focus should be on affecting opinion leaders’ attitudes (the more opinion leaders adopt the system, the 
lower the critical adoption rate for other system members will be) 

• Project manager and team should be open to external criticism 

• Project manager and team should continually question own assumptions 

• Project manager should continue to modify plan based on realities 

Institutionalization: 

Institutionalization means continuing the newly implemented change or stabilizing the use of an 
innovation (Sherry, 2003). In the institutionalization phase the innovation needs to be scaled and 
maintained. The following factors are considered important in this phase: 

• The organization should be fitted in a stable environment which is supporting and collaborating 

• The organization should have a relatively loose, centralized structure with good vertical 
communication channels 

• There should be competent leadership and management 

• The amount of change should be carefully dosed 

• User expectations should be managed: Innovation projects will fail if the users of a system are 
dissatisfied with it because it does not meet their expectations. Therefore, project managers should not 
only manage the development of the system, but also the perception of the system. 

• The innovation’s relative advantage as compared with the current practices should be communicated 

• Structures should be created that promote learning of new practices and observable incentive systems 
that support them 

• Effective communication among all parts of the system should be created 

• A high degree of observability, that is, a degree to which other persons than the innovator see its 
results as beneficial, should be created 

Since the educational innovations that are sponsored in the SURF ICT and Education Platform 
program take place within the time and funding limitations of SURF’s grant, and the sustainability of the 
innovations is considered a major goal by SURF, it seems necessary to require applicants to present a 
detailed plan for sustaining the innovation and to challenge them to realize this plan after the project has 
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ended, i.e. once the funding period of the innovation grant is over. To accomplish this, the success factors 
that are imperative for sustainable innovations to materialize, must be clearly communicated to the 
applicants/project managers. This can be done by using the Project Health Checklist. This Project Health 
Checklist (PHC-list) can be used during the different phases of the project by the project manager and team 
members in order to check the ‘health’ and progress of their project (i.e., are all important determinants for 
success taken into account in this project or do we need to adjust the project?). In addition, project 
managers and team members need to be familiarized with methodologies that can be used in the 
institutionalization phase to attain sustainable innovations. In this respect, CATWOE a Soft Systems 
Methodology initiated by Checkland and Coles (1990) seem promising. Furthermore, we believe that a 
financial incentive could be used to challenge the project team and the responsible educational institutions 
to maintain the innovation and diffuse it to other institutes. Currently, there is no financial incentive to 
sustain innovations. Lack of funding in this phase is certainly a disincentive, especially when adopting an 
innovation means that individuals must go through a learning curve and take on new responsibilities as a 
result of developing expertise (Sherry, 2003). Alternatively, it would be possible to make part of the 
funding conditional to the realization of the plan for sustaining the innovation. Regardless of which 
solution is chosen to promote the sustainability of an innovation, there must be a means to protect the 
organization if the innovation proves too costly to sustain. 

The identification of success and failure factors of educational innovation projects by means of 
literature study was the main goal of the first stage of this research project. In the second stage these 
‘objective’ factors will be used to analyze the SURF Education projects that have started in 1999, 2000, and 
2001 (see appendix I). More specifically, the plans and reports of these projects will be studied to 
determine if and how they have taken account of these success and failure factors, and to see if additional 
success and failure factors can be identified in these projects. In the third stage, the main people involved in 
the SURF Education projects will be interviewed to extend the ‘objective’ dimension of success and failure 
with a subjective dimension. It is expected that the overall results that emerge from these three stages can 
be used by SURF in their pursuit of successful projects with sustainable educational innovations. 

Although, this literature review has shown that the predominantly retrospective analyses of 
innovation projects can reveal interesting models, comprising numerous failure and success factors of 
innovation projects, there seems to be no firm empirical basis for these models. Therefore, it seems 
necessary to conduct more systematic research into the mechanisms that cause project success or failure. 

In the next phase of the project experts in the field of project management, project chairs of 
successful projects and evaluators/raters of projects will pinpoint factors of importance facilitative or 
detrimental to project outcomes and innovation implementation. After the interviews all potential success 
factors will be recorded and clustered using an affinity technique. The clusters will be labeled and 
clustered, creating a hierarchy of potential success factors. The project chairs will finally select the five 
most important success factors out of the hierarchy, and to rank their importance. This technique - the 
Experts’ Concept Mapping Method- is based upon Trochim’s concept mapping approach (1989a, 1989b) 
and was developed and perfected by Stoyanov and Kirschner (2004). 
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Abstract 

Instead of designing by following prescriptive models, students may benefit by approaching design 
as a process of free inquiry. This paper considers the impact of this inquiry-based approach on students’ 
thinking by considering the ways that students represented the design process at the end of an inquiry- 
based design course. 



Introduction 

When design is taught using prescriptive models (e.g., instructional systems design or computer 
integration into K-12 environment models), students come to understand what it means “to design” within 
the context of those models. That is, the model not only shapes the design process, but also models shape 
the designers’ thinking. What happens when student designers do not have such a model on which to 
depend? How do students construct an understanding of the design process when design is approached as a 
form of theory exploration and reflection, and thus not guided by prescriptive models but by student 
designers’ connections between scholarly literature and their own experiences, philosophies, and beliefs? 
This paper will explore these questions based on the author’s experiences teaching a “Principles of 
Instructional Design and Learning Technologies” course that culminated in students creating 
representations of design after completing inquiry -based design projects. I begin this paper with an 
overview of the course and the course design project. Then, I present the ways that students represented 
their design processes. In the last section of this paper, implications and conclusions are presented. 

Overview of the Course and the Design Project 

The “Principles of Instructional Design and Learning Technologies” course covers the history of 
instructional technology (cf.. Reiser, 2001a, 2001b), the appropriate role of computers and other media in 
the design process (cf., Clark 1983; Kozma, 1991), learning theory (e.g., behaviorism and cognitivism), and 
other seminal issues in the field. Historically, this course also provided an overview of prescriptive design, 
such as models of human-computer interaction design (e.g., Carroll, 1997) and instructional systems design 
(e.g., Morrison, Ross, & Kemp, 2004); but it was not a design course. Instead, this course was based on 
“typical” higher education pedagogy — lectures, group discussions of theoretical readings, exams, and 
research papers. Such an approach did allow students to learn about major principles and theories, but it 
did not provide students with a strong vision of how principles and theories influence the technology of 
designing instruction and/or facilitating learning — thus, the shift from “typical” pedagogy toward a learning 
through design (cf.. Nelson, 2003) approach. 

The design project for this course progressed in three stages. During the first three weeks of this 
course, students identified a design project and conceptualized their design approach. Since twenty-two of 
the twenty -three students in the course were full-time teachers, this initial approach was largely based on a 
“lesson planning” view of design. 

In stage two, students began designing and developing their project, but they were required to 
adjust their design approach to include decisions that were relevant to course content. That is, as each new 
topic was addressed during the semester, students adjusted their own design plan (and thus their 
understanding of what it means “to design”) to account for the topics. For example, when course readings 
dealt with extreme views in applications or theories for promoting learning (e.g., programmed instruction, 
on the one hand, and situated cognition, on the other hand), students had to either justify their previous 
decisions about their design in light of the readings, or they had to adjust their design decisions to create 
alignment with their understanding of the literature. Similarly, when students considered the Clark (1983) / 
Kozma (1991) debates about the role computers “play” in learning, students either had to adjust their 
previously -planned role of media within their design project or defend that role in light of interpretations of 
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the literature. The stated goal of the design project, then, was not to create “expert designers.” Rather, as 
the assignment guidelines noted, the ultimate purpose was for students to report how the topics covered in 
this course shaped their “thinking about” and “understanding of' design. 

In stage three, students had to report the ways that their thinking about design had evolved during 
the semester, and they had to offer a “representation” of what design “looked like” to them as a result of 
completing the course design project. Learning about design, then, resulted from personal descriptions of 
design, as opposed to resulting in students’ understanding of model-based prescriptions. Notably, design 
prescriptions were not covered in this course until students had largely completed their design project. The 
design projects, then, were influenced by students’ emerging understanding of theory, but not by formal 
prescriptive design. To the extent that design is problem solving, such a representation is useful in helping 
students concretely capture the problem space (cf., Jonassen, 2003) in which they had been operating. 
Furthermore, this representation provided students with a concrete artifact on which to reflect and through 
which they could consider how the course had shaped their thinking and understanding of design. The 
assignment guidelines noted that students’ descriptions might be “formal” and “academic,” but students 
were encouraged to take a more “creative” approach by, for instance, capturing the essence of design 
through an extended literary device (e.g., metaphor, analogy, or allegory), a mu ltimedia presentation, 
cartoon drawing, or even a sculpture. These “descriptive representations” will be the main focus of this 
paper, as they give the most insights into how students came to understand what it means “to design.” 

Students’ Representations of Design 

In this section of the paper, I summarize the various ways that students represented design and 
their design processes. Of the eighteen students who completed the course, four students represented their 
design process in the form of traditional-looking design models. One of these students offered a second 
representation of design (that was unrelated to the graphic) in the form of a metaphor. One student used a 
cartoon drawing that represented design metaphorically; the remaining thirteen students represented design 
through metaphors or analogies. In this section, I provide an overview of some of the representations. This 
overview of representations will serve as a basis for understanding an analysis presented in the next section 
of this paper. 

Four students represented the design process in the form of a traditional- looking model. One of 
these models dealt not so much with the content of the design process (i.e., learner analysis, writing goals, 
designing activities, etc.) as it dealt with the flow of the design process itself. The model used solid lines 
for linear processes, dotted lines to represent cyclical revision, and “nodes” to represent “dead ends” or 
stopping points. This student described ideas as being “set in motion,” “splitting,” “morphing,” and 
“branching” throughout the design process. Another student also used a traditional- looking model. This 
model was not “original” from the student; rather it was an adaptation of Morrison, Ross, and Kemp’s 
(2000) model. This particular student had previously taken an instructional systems design course and had 
used this model. His adaptation of the model included annotations, which came in the form of numbers 
written within the original model (e.g., he wrote a “2” next to the “learner characteristics” event inherent to 
the Morrison, Ross, and Kemp model). The numbers within the model corresponded to explanations of 
how the course had changed and expanded his personal understanding of the Morrison, Ross, and Kemp 
model. A third student who also had taken instructional systems design as a course also used more of a 
traditional model to represent design. Her representation involved various elements that traditionally are 
used within design models (e.g., goals and objectives, task analysis, choosing media, etc.). Interestingly, 
though, her model was unique in that she placed herself in the center of the model. From this 
representation, it can be inferred that she viewed design as not something to be looked at but something in 
which she was situated. Another student also used a traditional model. (Interestingly she also used an 
unrelated metaphor of design as a symphony orchestra, which will be discussed below.) Her model 
included the events of comparing the idea to her own philosophy and understanding of theory. But, more 
importantly, her model used two -directional arrows to represent the iterative nature of “trying idea on 
learners” (i.e., implementation) and “revision.” 

One student combined the idea of a “graphic” representation with a metaphor and accompanying 
cartoon drawing of six panels. She represented design as a transition from a “pretty” day; to a storm scene; 
back to nice weather, but with evidence of the storm in the form of an uprooted tree. The uprooted tree, she 
noted, represented the destruction of some of her previously -held notions about design. The remaining 
students used metaphors or analogies without accompanying graphic representations. For one student, to 
design was to sail on open waters. Sometimes, the “storm of learning theories” seemed too much to 
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handle; for example, a “gust of cognitivism blew hard” as a precursor to the “hurricane of constructivism.” 
When the storm wasn’t upon her, she felt herself “rocked back and forth” between two opposing currents: 
“Clark, Kozma, Clark, Kozma.” Interestingly, hers was not the only metaphor that involved water, as 
another student noted that “designing is . . . like jumping into deep, mucky, and swampy waters without 
knowing how to swim.” Similarly, another student noted that designing results in a feeling that you must 
“reach the top of the water to breathe again, but you do not have the strength to get you to the top. You can 
see the top of the water, but [you] are unclear how to get your body up there.” Another student also 
discussed design in relationship to water and tides: 

Throughout the design process I have felt like a fish just flopping around on the beach, hoping to 
survive long enough to make it back into the water. Just as I get close to the water, the tide comes in and 
pushes me further onto the beach. While flopping around, I meet numerous creatures that offer to help me 
back to the water; however, they really do not have the means to do so. Only the crab had the ability to bite 
me on the nose and pull me back to the ocean. 

Other students described design as a shopping trip, having ADHD, going through the stages of a 
butterfly’s life, being in a never-ending maze, an amusement park ride, hearing a symphony, or 
building furniture from raw materials. Admittedly, some of these metaphors were quite vague, 
where students did not fully explicate the metaphor itself. In other cases, though, the specifics 
were a bit more clear. One example of this clarity came from a student who compared design to a 

journey: 

Design is driving on a winding road. . . . [S]ome parts of the design journey are simple and 
refreshing, like driving on the straight, smooth section of road with the spring wind whirling in through the 
moon roof while sipping a fruit smoothie. Sometimes, though, you hit a pothole on the road. It throws you 
off-course. It may even cause you to pause and have to fix a flat tire. Or even worse, you may spill your 
smoothie and have to “clean” things up. And other times, the road ahead is curvy, so there are no surprises, 
and you see your course ahead of you. Design is like this path. ..sometimes smooth and easy, and yet other 
times it may be difficult and take more time as well as a break in action to contemplate or “fix” parts. 

Another student compared designing to quilting where one must find “a specific place for each 
patch, so that it all fits together appropriately.” This student notes that one occasionally “pricks her own 
finger,” but that “grandma” — in this case, the course professor — is always there to provide gentle guidance. 



Analysis of Representations 

Some themes did seem to emerge throughout the semester, and I considered those themes in 
constructing this section of the paper. Largely, though, it was while summatively assessing (i.e., grading) 
the course design projects that themes emerged. As I graded, I used the constant comparative method (cf., 
Glaser & Strauss, 1967) as a basis for finding themes. I begin this section by considering the nature of the 
designs themselves. Then, I offer more of a pragmatic analysis in light of course and project goals. 

Characteristic Analysis 

While collectively considering the nature of the design representations, three questions of interest 
struck me as worthy of consideration as a basis of analysis. Each of these questions will be discussed in 
turn, and readers should see these questions as ones that brought forth a thematic interpretation of the 
various design elements. 

The first question can best be stated thusly: How many were representations that came in the form of 
“natural” processes as opposed to processes that are more indicative of the artificially -contrived? Such a 
question is useful to the extent that design is a science of the artificial (cf., Simon, 1969). Addressing such 
a question might provide insights into the ways that students viewed design as something that they can 
construct and control. The students who created traditional looking models obviously were creating a 
representation that was artificially contrived. Table 1 divides the metaphorical representations into the 
categories of “natural” and artificial. 

Natural Processes Artificial Processes Hybrid or ambiguous 

• Having ADHD • Shopping • Amusement Park 

• Fish flopping on the • Cooking Ride 

beach • Quilting • The elements of an 

• Stages in a • Building furniture orchestra 
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butterfly’s life • Sailing 

• Drowning 
• Transitions from 
peaceful to stormy day 

Table 1 . Design metaphors by nature of process. 



• Driving on a winding 

road 

• Jumping into murky 

water 

• Being lost in a maze 



To some extent the above categorization of these design representations suggests a view of design as either 
something the designer can control or that the designer is controlled by. For example, the notion of 
designer as a fish flopping on the beach is a metaphor of being controlled by “nature.” The author of this 
particular metaphor even notes that tides pushed her further away from the water and towards the beach. 
Similarly the notion of trying to avoid drowning by aiming to “reach the top of the water to breathe again” 
suggests that design is a natural instinct of combating other natural occurrences. On the other hand, some 
metaphors suggest that students viewed designing as a process over which they had control. The metaphors 
of sailing, cooking, and quilting, for example, imply that designers can actively make decisions and control 
the process as an artificial science that can be well-executed. 

A second question also arose for me during this course and, more specifically, during summative 
assessment of the course projects: To what extent did the students’ representations indicate the non-linear 
nature of design? Some of the design representations are clearly linear. For example, the student who 
suggests that designing is like going through the stages of a butterfly’ s life provides a representation of 
design as a linear process. In fact, she implied a feeling of helplessness as certain stages took their natural 
course. Furthermore, comparing design to a ride at an amusement park offers a view of design with a 
starting point and direct movement toward an ending point without much control by the designer. It is only 
coincidence, it would seem, that this student found some resolution to her design dilemmas as the “floor 
[was] beneath [her] again, as the ride [came] to an end.” That is, her resolution did not come as an act of 
going through a non-linear process; rather, the ride (i.e., design) had a natural end to it and that end was 
beyond her control. 

On the other hand, some of the representations suggest a non-linearity to them. As I noted in 
describing the representations, one “traditional” model depicts the non-linearity of implementation and 
revision. The student who adapted the Morrison, Ross, and Kemp (2004) model also noted the non- 
linearity of that model and its manifestation within his course design project. Some of the metaphorical 
representations of design also suggest a non-linearity within design. For example, the student who used 
“drowning” as a metaphor for design discussed the process of trying to rise to the top but then being pulled 
under again. Furthermore, both cooking and building furniture are non-linear. In the process of cooking, 
for example, the chef is, in the student’s words, “trying to perfect . . . recipes [through] a continuous 
process to achieve the ultimate masterpiece. . . . [I]t takes a lot of mistakes to get to a masterpiece.” 

A third question deals with considering the degree to which the student designers understood 
themselves as designers: How do the design representations portray the designers themselves? Do they 
situate themselves within the process? Or do their representations indicate that they are apart from the 
process? This particular question struck me as I looked at one of the more traditional models of design that 
a student created. Her model was circular much like the Morrison, Ross, and Kemp (2004) model of 
Instructional Design, but in the center of the circle, she included a representation of herself. She, then, was 
within the design looking outward. I found this compelling, and began wondering whether other 
representations of design mainly had the student on the “outside” looking in, thus design served as more of 
a cognitive plan; or did the representation include the learner as situated within the design processes? 
Admittedly, such a question may overlap with previous questions. Still, it seemed to me that such a 
consideration is worthy. 

Several students did situate themselves within the design process. Consider as examples, the 
students who used analogies of boating, riding an amusement park ride, and driving on a winding road. All 
of them are in the “seat” that is seminally situated within the process. Interestingly, there are key 
differences, as the captain of the boat and driver of the car are in control, but the rider at the amusement 
park is not. So, two are situated in positions of power, while another is situated almost as a victim. Others 
situated themselves as “creators” within the process. Consider the students who used analogies of cook, 
quilt maker, and furniture maker. They situated themselves as creators of the design process. 

Conversely, others situated themselves outside of their own design representation. Consider the 
student, for example, who used the cartoon drawing from a peaceful to a stormy day. She was not situated 
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within the cartoon. Rather, she was looking in at the cartoon. Similarly, the student who annotated the 
Morrison, Ross, and Kemp (2004) design model was not situated within the design model. The model was 
presented as something to be analytically viewed and considered — it presented a cognitive plan, not a 
situated learning representation (cf., Cobb & Bowers, 1999). 

Functional Analysis 

What was the “function” of these design representations? Were they successful in meeting that function? 

A full defense of the purposes of this design project is beyond the scope of this paper. Nevertheless, a 
functional analysis can only be discussed in light of the goals of this project. In short, I was trying to 
achieve some of the following: 

• Enabling students to see design through a process of metaphorical synectics (cf., Joyce & Weil, 
1992) as metaphorical thinking can help people see the familiar in a new light (Lakoff & Johnson, 
1980). 

• To promote a design environment where reflection is not “de-coupled from design activity 
(Shambaugh, 2004), but rather where iterative cycles of design and reflection inform each other. 

• To promote a “learning by design” (cf., Edelson, 2002; Nelson, 2003; Nelson & Knowlton, in 
press) approach where the act of designing results in a new way of knowing. 

Some students struggled to “break the habits” of the curriculum-development approach to design 
that is typical of their experiences as K-12 teachers. More broadly stated, students were shaded by previous 
design experiences. Those who had already taken more “formal” design classes — such as “Instructional 
Systems Design” or user-centered design — struggled to make design decisions without reference to the 
models with which they were already familiar. The most clear example of that is the design model that was 
an annotated representation of the Morrison, Ross, and Kemp (2004) design model that I described earlier 
in this paper. Other examples of this abound, as well, though. Earlier in this paper, I described a model in 
which the student situated herself in the middle of the design process. Still, her representation of design 
was similar to the Morrison, Ross, and Kemp model. 

Even where students did successfully represent design in a way that was perhaps new to them — 
such as design as riding an amusement park ride or design as a fish flopping on the beach — most students 
struggled to integrate their novel representation throughout the remainder of the paper. Typically a student 
would introduce the metaphor in a paragraph, but then they would leave the metaphor altogether to discuss 
their understanding and view of design. So, while some metaphorical activity was present; that metaphor 
did not “shape” their explanation of design throughout the rest of the paper. In fact, as I have already noted 
one student offered a metaphor of a symphony but then offered an unrelated “traditional” model. Another 
student created an extended metaphor in her paper, but then when discussing her new understanding of 
design brought in numerous additional metaphors in the form of one liners (e.g., you can lead a horse to 
water, but you can’t make it drink). To some extent, these variations defeat the entire point of metaphorical 
thinking and undermine the bulleted list of goals that I present at the opening of this section of the paper. 

Implications, Conclusions, and Forward Progress 

In spite of a few of the shortcomings of this project, I do think that using an open-ended inquiry- 
approach to design has merit as suggested both through the literature (cf., Edelson, 2002) and through an 
analysis of the design representations in this course. Particularly in light of the fact that the program in 
which I teach attracts k-12 teachers, using an inquiry -approach to design serves as a nice transition from 
their “teacher education” way of thinking to a more systematic design approach. Decision makers in other 
graduate programs might sequence courses such that students learn about design in general prior to taking 
prescriptive design courses. 

Furthermore, focusing solely on this course and its future, I will “stay with” the general design 
project described here. Based on some of the results from this initial implementation of the project, I find 
myself wondering how I could more strongly scaffold students’ efforts on two levels. One level would be 
to better scaffold students’ understanding of the design project itself. How can an instructor of such a class 
help students see the benefits — largely metacognitive benefits — of engaging in metaphorical and 
representational thinking? One way would be to add a literature base to the course that raises the notion of 
situated cognition as a perspective from which we learn. Perhaps sources, for example, that deal with 
“learning by design” (e.g., Davis, 1998; Hoadley, 1997, Rittel, 1984) would be useful in promoting such a 
perspective. 
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A second level would be to better scaffold their production of a “representation” and discussion of 
that representation. This might involve shaping other elements of the course to directly support the design 
project. For example, currently in this course, there are three captions of grades: the course design project, 
in-class participation, and summary/reaction journals. (See Knowlton, Eschmann, Fish, Heffren, and Voss, 
2004, for a discussion of summary /re action journals within this particular course.) Shaping course 
activities to more directly include processes that would help students make decisions about their design 
representations might be useful in scaffolding students’ thinking about design and how to represent it. For 
example, during small group discussions about course readings, I might more directly prompt students to 
bring their design projects into the discussion as a basis for understanding a reading’s implications on 
practice. Furthermore, my comments within their summary/reaction journals might provide a forum for me 
to urge further thinking about their design representation. Using other areas of the course would also better 
help students see the ways that the course could be synthesized into a useful experience. 
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Using Asynchronous Discussion to Promote Collaborative Problem 
Solving Among Preservice Teachers in Field Experiences: Lessons 
Learned from Implementation 

Dave S. Knowlton 

Southern Illinois University Edwardsville 

Abstract 

Designers who are charged with creating online environments that support students’ problem- 
solving efforts and instructors who are searching for instructional strategies to support students ’ 
asynchronous problem-solving efforts will find this paper useful. The paper describes a systematic 
approach for using computer-mediated bulletin boards to support collaborative problem solving among 
students infield experiences. This paper also briefly describes lessons that the author learned as he 
implemented the approach in a teacher education setting. 

Introduction 

The overall purpose of this paper is to share specific instructional strategies and assignment 
guidelines that promote the sound use of computer-mediated bulletin boards (CMBBs) to facilitate 
collaborative problem solving among students who are engaged in field experiences. To reach this overall 
purpose, the paper will proceed in three parts: First, theoretical connections among the purposes of field 
experiences, the methodology of problem-based learning (PBL) and the sound educational use of CMBBs 
will be made. Second, a format for a CMBB discussion that is congruent with these theoretical connections 
will be offered. Emphasized within this format will be specifics from the assignment guidelines that serve 
as instructional strategies to help students maximize the educational potential of the discussion. Third, I 
offer “lessons learned” from one implementation of this approach. Inherent to this description of 
implementation will be design and pedagogical advice for others who may attempt to employ the approach 
that I describe; also, within the “lessons learned,” suggestions for future research will be made. 

Theoretical Connections among Field Experiences, PBL, and CMBB Discussion 

Discussion via CMBBs can support field experiences (Doering, Johnson, & Dexter, 2003); 

CMBBs also can support certain types of problem solving (Uribe, Klein, & Sullivan, 2003; Jonassen, 2002; 
Jonassen & Kwon, 2001). Beckett and Grant (2003) have made cursory connections among CMBBs, PBL, 
and field experiences. Scarce — though not non-existent (cf., Knowlton, 2004) — within the literature are 
firm theoretical connections among all three. The argument for supporting all three is based on two 
premises : 

Premise #1: Students must solve problems to be successful in field experiences. 

Premise #2: CMBBs are an efficient tool for solving many types of problems. 

If these two premises are true (and each will be discussed briefly below), then a conclusion follows: CMBB 
discussions may be useful as a tool for solving problems that students encounter as they participate in field 
experiences. 

Solving Problems as Success in Field Experiences 

When students move from classroom settings to field settings, they will experience a shift in what 
it means “to learn.” In classrooms, learning is often the result of memorizing and regurgitating a database 
of information. Even when professors try to incorporate problem solving into classrooms, those efforts are 
often inferior because the problems are well-structured and little — if any — real utility exists in solving 
those problems. 

In field experience settings, though, the success of the student, who has now been recast as human 
capital, depends on that student’s ability to solve real problems that are often ill-structured. These 
problem-solving efforts become the very definition of learning within field experiences; thus, students must 
solve problems if a field experience is to result in learning. This view is consistent with Jonassen (2002) 
who notes differences between classroom learning and learning in the “real world.” The learning results 
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from the confluence of, on the one hand, the variety, frequency, and complexity of the problems and, on the 
other hand, the high “stakes” of not adequately solving the problem. 

CMBBs as Tool for Efficient Problem-Solving 

Jonassen’s (2002; 2003) point that not all online learning environments lend themselves to all 
forms of problem solving is clear. Still, CMBB discussion among students is a useful strategy for 
analyzing and solving many types of problems. 

CMBB discussion, for example, can support many problem-solving processes through “writing to 
learn” (cf., Lindeman, 1995). As students represent problems in written form — a process advocated within 
many problem-solving models (e.g., Abel, 2003; Jonassen, 2003) — they are making the problem concrete, 
turning the ineffable into that which is effable. “Seeing” their problem and their own ideas about their 
problem can better help students develop a useful perspective on the problem. 

Beyond the “writing to learn” standpoint, CMBB discussion increases opportunities for social 
learning and collaborative thinking — for cognition to be distributed across a community of learners who are 
engaged in field experiences but in different contexts. Students in field experiences are estranged as a 
result of both distance and time. In isolation, these students are situated in a specific context, but 
particularly for students who are accustomed to traditional classrooms, the estrangement can be 
disconcerting. When students are embedded within a specific situation or context, their thinking becomes 
both bound by and free within that context — their cognition is situated. Students recognize themselves as 
situated because they accept “the mutual relation of content and context, of individual and environment, 
and of knowing and doing” (Barab, MaKinster, Moore, Cunningham, & the ILF Design Team, 2001, p. 

73). 

Perhaps there are varying degrees of being bound within a context as one’s participation moves 
from the periphery of that context to the center of meaningful activity (cf.. Lave & Wenger, 1991) and as 
one comes to understand the fluid and iterative relationship between plans and actions (cf., Suchman, 

1987). This is true for students in field experiences. While such situatedness is a benefit to students 
because it gets them beyond the walls of an artificial classroom, it also is a hindrance because some of the 
benefits of classrooms — discussion and collaboration among students, for example — are lost. In short, 
students in field experiences can benefit from thinking within a context, but they need to participate in a 
type of distributed cognition in order to fully appreciate the uniqueness of their situation. Discussion 
through CMBBs can support this shift from situated to distributed cognition. 

CMBB Strategies and Guidelines for Enhancing Field -Based Problem Solving 

I have noted that students must learn to solve problems in order to be successful in field 
experiences. I have further noted that CMBBs can be a useful medium for problem-solving in field 
experiences but only if the use of the CMBB supports students as they transition from “writing to learn” 
(representing the problem for themselves) to thinking within an online community (experiencing 
distributed cognition). A CMBB discussion assignment must be designed to support this transition. 

Slightly less lofty, yea though just as important, a CMBB discussion assignment must help 
professors maintain a delicate pedagogical balance: Professors must not usurp students’ authority by 
solving the problems for them, or even providing a close-ended structure that leads students to a finite 
range of solutions (Beckett & Grant, 2003); yet professors must provide enough scaffolding so that students 
must — as described above — make their problems concrete and engage in social interaction as a means of 
solving the problem. In this section, assignment guidelines are offered that can help professors maintain 
this balance. These guidelines are offered not as a rigid approach to be denotatively followed, and 
professors might find that some deviation from the suggestions here will be useful in implementation. 

The assignment guidelines are similar to those already existing in the literature (cf., Knowlton, 
2002). Participants should be divided into two groups and discussion should be conceptualized as three- 
week cycles of sharing and response. At the end of each cycle, roles should be reversed so that group one 
participants perform the responsibilities of the participants in group two and vice versa. As it is not the 
medium of delivery that creates learning (Clark, 1983, 1994; Morrison, 1994), an emphasis on instructional 
strategies is emphasized (cf., Morrison & Guenther, 2000) within an explanation of each week of the 
discussion cycle. Notably, though, professors can provide resources that help participants across all weeks 
of the discussion. For example, professors might offer general guidelines for making CMBB discussion 
useful (cf., www.siue.edu/~dknowlt/DiscussContributes.htm). 
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Week One of the Discussion Cycle 

Writing to learn and problem identification are the two areas being emphasized within week one. 
Participants assigned to group one are responsible for describing a “problem” that they are experiencing 
within their field experience. Following the guidelines for a “critical incident analysis” (Brookfield, 1987, 
p. 49), participants might be required to include numerous elements within their problem description. 
Professors can guide participants toward including of these elements by providing guiding questions as the 
basis of a week one contribution: 

• When and where did the incident occur? 

• Who was involved? (This should be phrased as roles — e.g., “my manager” or “a customer” — 

rather than names.) 

• What was it about the incident that made it problematic? 

The strategy of a critical incident questionnaire obligates discussion participants to organize the 
problem for themselves, which may result in increased clarity of the problem definition and its scope. This 
general approach is more likely to help participants better move toward productive solutions (Bruer, 1993). 

One strategy that may be useful is to specify the type of problem that is worthy of being 
considered within the discussion. Within the context of a corporate internship, for example, problems 
might come in the form of those that are interpersonal (e.g., a conflict with a co-worker); ethical (e.g., 
questionable accounts receivable practices within an accounting department); or managerial (e.g., a lack of 
support from middle management to adequately solve a problem). Initially, it is useful to give students 
broad reign over the types or problems worthy of discussion, but if a certain category of problem 
monopolizes discussions, professors might specify types of problems that should be given priority as 
students decide what problems to share. 

Week Two of the Discussion Cycle 

Participants in group two are responsible for representing the problem through both retelling and 
elaboration. See Mayer (2002) for a discussion of the value of restating a problem and Johnsey, Morrison, 
and Ross (1992) as well as Jonassen (1998) for a discussion of the role of elaboration as an instructional 
strategy. The type of representing that I describe is a framing of the problem using both practical 
experiences and course theory. In terms of practical experiences, certainly expert problem solvers tend to 
relate current problems to other problems that are analogous in one way or another (Lakoff & Johnson, 
1980). Therefore, there is value in when a participant responds to a problem by sharing a parallel or 
similar problem from another context. 

Participants should also be required to help frame the problem using resources beyond their own 
experiences, though. Such resources will help those in field experiences connect their problems back to the 
type of textbook theory that they studied prior to their field experience. This type of connection is 
supported by Beckett and Grant (2003). Through indexes and tables of content, textbooks become a 
learning-on-demand resource, where participants self-select readings that would most likely contribute to 
an analysis of the problem-at-hand. In essence, this approach is indicative of the “classic version” of PBL 
(Hmelo & Evensen, 2000), which requires students to collaboratively “formulate learning issues by 
determining factors that may contribute to the cause or solution of a problem” (Knowlton, 2003, p. 5). 

Week Three of the Discussion Cycle 

More practically, all participants in both groups one and two should be responsible for three 
contributions to the CMBB discussion during week three of the cycle. Importantly, assignment guidelines 
should specify that week three contributions should be replies to week two contributions, not replies to the 
original problem discussed during week one of each cycle. This criterion, while initially may seem overly 
bureaucratic, is designed to create interaction among the participants within the CMBB discussion and 
promote a deeper analysis of the issues embedded within the problems, not just continued (and often 
redundant) “solutions” to the original problem. Pena-Shaff, Martin, and Gay (2001) determined that most 
participants “read and constructed their comments based on other participants’ messages. However, [they] 
were unable to find explicit collaboration [among CMBB discussion] participants” (p. 54). By delineating 
some sequence to the discussion (i.e., insisting that week three contributions respond to week two 
contributions), the present approach is more likely to move participant collaboration toward a broader 
understanding of the problem, and thus more solutions. 



377 




The assignment guidelines that participants are given should clearly specify the purpose of week 
three contributions — to “further define the problem-at-hand through dialogue” by “reading what [others 
had written] within a ‘thread’ of discussion and interact by responding to [each other’s] ideas.” The 
wording of these instructions shifts the emphasis from cognition to distributed cognition. Such a shift 
would lead the “the unit of analysis” for understanding knowledge construction from “the individual’s 
cognitive system” to “the interrelatedness of a diversity of intellectual systems” (DeHaan, 2002, p. 33). 

The wording used is equally important to the extent that it helps participants understand their 
responsibilities to other participants. 

To scaffold students’ abilities to offer week three contributions that do move the conversation 
forward and toward the type of interrelatedness that I describe, professors might offer a list of discussion 
prompts that participants might use as the basis of their week three contributions. These discussion 
prompts are often similar to the following items: 

• Pick two replies to the same problem and discuss why you think one would work better than the 
other. 

• Pick a reply to a problem and discuss the strengths and weaknesses of the proposed solution 

• Pick a theory that someone mentioned as a help to understanding week #2 and apply that theory 
differently (or more thoroughly). 

• Discuss your experiences with how a solution has/has not worked in the classroom. 

• Write a summary of responses to your own problem and describe what the biggest things that you 
are taking away from your problem are. 

Notably, I urge participants not to feel bound by these prompts, but such prompts can serve as a basis for 
helping participants see approaches for using week three contributions constructively. 

Lessons Learned from Implementation 

The approach and advice for implementing CMBB that was presented in the previous section of 
this paper seems to meet the goals of problem solving in field experiences. Across each week specific 
strategies are used to promote movement towards a better understanding of select problems. I have 
implemented this approach in the context of a two-year, field-based teacher-certification program that was 
designed to prepare undergraduate students (N=60) for careers as public school teachers. These preservice 
teachers were assigned to K-12 classrooms in partnership schools. During the first year of the two-year 
program, the preservice teachers often served as a teacher’s assistant, paraprofessional, or aide. During the 
second year, though, the preservice teachers’ became more centrally involved in teaching and learning 
activity. In this section, I describe lessons that I learned through this implementation. Inherent to these 
lessons is additional advice to others who may wish to use the approach presented in this paper. 

Lesson #1: Shaping Discussion around Problem Solving Processes is Not Enough 

As can be seen from the discussion of each week within the cycle, the assignment is shaped 
around notions of good problem solving. For example, neither weeks one nor two seem to emphasize 
solutions. Rather, they emphasize students’ carefully defining the problem and raising learning issues — an 
emphasis consistent with many models of problem-solving (Knowlton, 2003), including, for example, the 
classic version of problem solving (Hmelo & Evensen, 2000). Perhaps instructional strategies that aim 
students toward habits of good problem solving are not enough. The preservice teachers in my 
implementation, for example, didn’t seem to be aware that each week was following a problem-solving 
model. Rather, the participants were denotatively following instructions within the assignment guidelines. 
So, were students engaging in problem definition and representation as purposeful activity that served as a 
precursor to developing collaborative solutions? Or, were students simply fulfilling an assignment? 

Certainly, participants’ motivation is one factor that determines their willingness to engage in 
meaningful problem solving. But professors might help students understand the ways in which the 
approach presented in this paper does guide them through the problem solving process. Through such 
efforts on the professor’s part, students may come to realize that participation in CMBB discussion is not 
“busy work” and does have an educational intent. As a result, students would be more equipped to not only 
participate in discussion but also to reflect on their participation and think metacognitively about their 
discussion participation. 

Lesson #2: Balancing Issues of Breadth and Depth can be Problematic 
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My experiences with implementation suggest that the approach presented in the previous section 
of this paper does a good job of guiding participants toward defining the problem and developing related 
issues that may influence the problem or the solution. This type of depth is useful and supported as a 
hallmark of problem-based learning (Hung, Bailey, & Jonassen, 2003). However, I struggle with two 
seemingly contradictory elements regarding the issue of breadth versus depth. 

Is there too much breadth? During week two of the cycle, participants delineated and discussed 
related issues, theories, and analogous problems as a tool for helping to shape the problem that had been 
presented in week one. In all honesty, I often struggled to see the connections between the issues raised in 
week two and the problem-at-hand. For example, it would not be uncommon for the problem offered 
during week one to be related to a learning issue (e.g., many students didn’t meet the objectives in a lesson 
plan). But then, during week two, discussion dealt with cultural or economic reasons that parents were not 
actively involved as students completed homework. Certainly, a lack of parent involvement and classroom 
management can negatively affect the efficacy of a lesson, but are these issues the most salient given the 
problem-at-hand? Because I was sometimes torn as to the relevance of week two “issues” that were raised, 

I was tempted to offer more scaffolding to narrow the range of issues that were relevant. Doing that 
without undermining the very types of student authority and initiative that I was trying to build, though, is 
difficult. Development research might focus on the impact of such scaffolding. 

Does the approach described in this paper allow for enough depth? My experiences suggested that 
because the preservice teachers spent two weeks defining the problem (week one of the cycle) and bringing 
up related issues both from textbooks and their experiences (week two of the cycle), there was difficulty 
during week three in coming to “solutions.” Often during week three the preservice teachers simply 
offered more analysis of the problem and its scope. I attempted to overcome this problem by assigning a 
related — but different — assignment where the student summarized the discussion that resulted from his/her 
problem and created a solution that she/he planned on implementing. 

Lesson #3: Balancing Facilitating with Grading is Difficult 

Educationally useful is the idea of professors facilitating and managing CMBB discussions to help 
students maximize their own educational benefit. This educationally -useful activity sometimes becomes 
secondary to the administrative realities of grading. In the implementation that I reflect on in this paper, I 
became bogged down in grading, rather than facilitating the conversation in ways that would have been 
educationally useful toward the preservice teachers’ continued good thinking and problem-solving habits. 

In fact, I find myself wondering if a stronger shift towards facilitating would have helped overcome the 
problems that I identified in “Lessons” one and two as described above. 

Professors might overcome this problem by shaping assessments so that peer and self-assessment 
help students make judgments about their own contributions (and even determine grades). Self- and peer- 
assessments can come in forms as simple as dichotomous or Likert -scale checklists, but they also might 
involve processes indicative of a more careful analysis of students’ contributions, such as qualitative and 
open-ended assessments. The emphasis on peer and self-assessment would “free up” the professor to guide 
and lead the discussion. The problem with this advice, of course, is that grades based on student 
assessments will possibly be inflated or will not be an accurate measure of student performance within the 
discussion. 

Another way to overcome this problem of grading working against meaningful guidance might be 
to use a form on which students self-report their participation at the end of each three-week cycle. For 
example, the form might ask students to list the subject line of the threads in which they participated. The 
form also might ask students to briefly list the resources that they used in theoretically framing a problem 
to which they were responding. This form, then, would not be a self-assessment as much as it would be a 
productivity report. However, when students submit their form, the professor will have a list of threads in 
which to find their contributions. This might make the process of “grading” less time consuming. 

Conclusions 

While the implementation that I describe is within a teacher education setting, the strategies and 
scaffolding that I describe in this paper are not bound by content or discipline; instructors across disciplines 
who supervise field experiences might find the approach to discussion that is described in this paper to be 
useful for facilitating strong discussion. Importantly, the literature used to support this approach is a 
synthesis of literature from each “area” discussed here — i.e., PBL, bulletin board discussion, and field 
experiences. It is hoped that readers find this synthesis of ideas pedagogically useful. 
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Abstract 

This is a discussion about the role of a case or authentic scenario in an online learning 
environment. Using authentic cases for intentional learning promotes effective, cognitive, and affective 
transfer between learning space and performance space. Creating an online case event provides an 
opportunity for learning design professionals to use instructional systems design in an authentic, team- 
oriented, web-based learning environment. A case approach ends the Instructional Systems Design ( 1SD ) 
learning process and helps to facilitate further research of online learning environments. Case studies 
have been an effective tool for developing professional knowledge across disciplines, however, case events 
dedicated to the study and practice of instructional design is limited among learning sendees professionals. 
The purpose of this discussion is to offer educational benefits about learning and practicing instructional 
design within an online learning environment. 

Introduction 

There are too few opportunities for learning service professionals and students to practice 
authentic instructional design as a part of their respective training and academic preparation. The 
professional practice of instructional design (ID) requires high-level problem solving, critical thinking, and 
interpersonal skills; design problems are often complex and multi-dimensional. Novice instructional 
designers face practical issues for which they were not prepared (Julian, Larsen, and Kinzie, 1999). 
Analyzing cases provides an opportunity to explore professional issues while students leam (Kinzi, Hrabe, 
and Larsen, 1998). Learning through a case-based environment allows novices to analyze the case, reflect 
on relevant theories and techniques in attempting to understand a real problem, develop a response, and 
consider potential consequences. 

Over the past several months, we have explored aspects of an educational approach using an 
online case event that served to provide designers with an opportunity for teamwork in an authentic 
environment. The components of this discussion include: 1) a review of the case method as a learning 
strategy, 2) the role of a case in an online learning environment, and 3) learning instructional design from a 
case experience. This discussion concludes with recommendations for using case method for a successful 
online learning environment. 

The Case Method as a Learning Strategy 

Case methods help students examine theories during the learning process, and to apply these theories to 
situations they may encounter when they are no longer students. Educators and trainers should regard a 
classroom as any learning space, performance spaces of the information age can be situated at remote sites, 
accessed at convenient times, and personalized to match the capability of individual learners. Each episode 
of guided learning is distinctive and separate, while remaining part of a larger curricular scheme. Using 
case methods and an online learning environment, students ’ learning space becomes larger than a 
traditional classroom and is similar to performance space. 

Case events help learners to think like practicing professionals by seeking solutions in realistic 
situations. Case events have been used extensively in professions such as law, business, and medicine and 
more recently have gained popularity in other professions such as teacher education, engineering, nursing, 
and instructional design (Ertmer &Russel, 1995). Merseth (1994) advocates case study methodology for an 
appropriate tool for teacher education. Merseth states that a case study is a descriptive research document, 
often in narrative form that is based on an authentic situation. According to Merseth, “there are three 
purposes for using cases: 1) cases as exemplars; 2) cases as opportunities to practice analysis, the 
assimilation of differing perspectives, and the contemplation of action; and 3) cases as simulations for 
personal reflection’’(1996, p.3). Case studies provide realistic scenarios that encourage students to 
recognize the link between theory and practice, weigh resource constraints in choice alternatives, practice 
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analytical skills, and develop related skills such as group decision making and communication (Rupe, 

1993). Thus, case-based approaches help to close the gap between what happen in the classroom and the 
expectation of the reality outside the classroom in the performance space. 

The Role of a Case in an Online Learning Environment 

Internet technologies provide an effective vehicle for delivering cases to students, and other case 
event participants . The use of the Internet provides case materials and online environments for case 
discussion. Case studies are now available online and case discussions can be conducted using 
asynchronous and synchronous online discussion tools. Perry (2000) claims the “Internet and multimedia, 
which includes the non-linear integration of video, audio, graphics, and text, can provide a rich 
environment for case studies that promote the construction of knowledge about integrating technologies 
into the curriculum of a learning community of peers and faculty facilitators’' (p. 4). The online 
instructional design case event discussed here subscribes to Perry’s position that an online case experience 
promotes an integrated, action learning approach with the potential to increasing effective practice. 
Kovalchick, Hrabe, Julian, and Kinzie (1999, p.145) suggest that the web provides three significant 
capabilities for the delivery of case studies: 

1. The ability to simulate real-world complexity, 

2. The ability to use multiple media in case presentations, and 

3. The ability to use hyperlink/ hypertext navigation features. 

Presenting case studies on the worldwide web (WWW) offers additional educational benefits, 
including incentives to learn, and new and flexible learning methods (Hayden & Ley, 1997). The Internet 
provides a unique environment for presenting case studies that allows users to gather information, identify 
issues, create solutions, receive feedback, and gain experience through problem solving. The interactive 
nature of the WWW and e-mail encourages students to consider various perspectives and possible solutions 
and to develop a rationale for decision making while enabling worldwide discussions. 

Case events allow students to construct knowledge in authentic environments, assume personal 
responsibility for learning, and work cooperatively to produce something of real value (Grainger, 1996). 
Case experiences also provide students with genuine opportunities to experience the risks and 
consequences associated within a defined context, allowing students to explore professional issues while 
they are still learning about design. Kinzie et al. stated, “cases should offer enough depth and complexity 
to provide realistic challenges” (1998, p.55). More specifically, case based competition is beneficial to 
learners because knowledge and skills are best learned in contexts that reflect the way they will be useful in 
various realities. Learning in case-based environment allowteams to analyze a case, while experts in the 
field pose questions, evaluate case responses, and contribute their own perspectives. Online case-based 
events are student -centered, active learning experiences where students can reflect on relevant theories and 
techniques in attempting to understand a genuine problem, develop a response, and consider a variety of 
acceptable responses to a given situation. 

Learning Instructional Design from a Case Experience 

Instructional design is often learned informally on-the-job or as an isolated, stand alone college 
course. However, recent evidence suggests an action learning approach as an effective strategy for 
becoming proficient in the practice of instructional design (Bannan-Ritland, 1999). The principles of an 
action learning approach provide a framework for re-examining methods of teaching of instructional 
design. Components of action learning correlate closely with the specific challenges involved in teaching 
the ill-structured, complex problem solving processes of the practice of instructional design. An action 
learning approach provides instructional design practice in businesses, schools, and government as a part of 
their respective training and academic preparation prior to assuming employment as an instructional 
designer. 

Action learning is an instructional strategy used in training and education to increase the fidelity 
between learning performance and job performance. The problem is that many training tasks are de- 
contextualized and incongruent with workplace realities, thus having a low fidelity between what occurs 
during classroom preparation and what is expected on the job. 

Action learning is a performance-oriented, learner-centered, instructional strategy. The purpose of 
action learning is to promote immediate and long-term learning transfer. Action learning is facilitated 
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through a coaching model approach where an experienced person is partnered with an individual or team 
and the coach is available to answer process questions during the training and for a designated period after 
completion of the training. While action learning incorporates several learning theories, five attributes are 
necessary for an instructional strategy to be considered as an Action Learning Approach: 

1. Active 

2. Interactive 

3. Situated 

4. Authentic 

5. Case-based 

Authentic instructional design should emphasize learning objectives that represent the training 
context while on-the-job objectives represent the actual performance context. Learning objectives describe 
what the learners will be able to demonstrate with the learning environment. However, the learner’s 
performance in the classroom or at the computer in a computer-based training environment is not 
necessarily the same as the on-the-job performance. Learning on simulators, models, through hands-on 
practice, role-play scenarios, and actual equipment and tools, provides more effective training than learning 
exclusively from lectures, demonstrations and other passive techniques. 

We have explored aspects of this educational approach during the past year, by constructing an 
online case event (http://www.itcaseevent.com/), at the Educational Psychology and Instructional 
Technology Department, University of Georgia. We believe such an online case event will provide an 
opportunity for participants to practice instructional design in an authentic, team-oriented, online learning 
environment. The process is a rewarding, challenging and complex endeavor. A case needs to be created 
that illustrates an authentic problem requiring an instructional design solution. While developing a case for 
an event, consideration should be given to the fact that novice instructional designers require a genuine 
problem to reflect the situations that they would encounter in the workplace, or approximate this 
environment as closely as possible. The intended audience for the event needs to be determined. The 
rationale for authentic learning is that learners can only realize the utility of the concept being taught by 
focusing on an authentic problem and hence, providing an authentic solution 

Recommendatio ns 

This discussion has focused on ways in which online case events provide opportunities for 
learning service professionals to engage intellectually in solving authentic instructional design problems. 

We presented ways case methods provide an opportunity for the case event participants to discuss 
professional issues about instructional design in an online learning environment. We discussed about how 
online case events provided an opportunity for participants to use instructional design in an authentic, team- 
oriented, web-based learning environment. 

We believe case-method teaching and provided learning strategies promotes discussion of 
complex educational dilemmas, reflection on successful learning strategies, and ways to evaluate, 
summarize, and communicate perspectives about the shared case experiences of online case event 
participants. Based on our experiences thus far, we offer the following recommendations: 

1. The case event should illustrate a genuine and authentic problem that requires an instructional 
design solution. 

2. The case event should be designed to challenge participants to demonstrate knowledge of 
instructional design principles and this exercise should incorporate the problem that was 
presented. 

3. The project manager for the online case event should be chosen based upon ability to generate a 
plan for communication between all stakeholders and excellent time management skills. 

4. Careful preparation allows design teams to anticipate obstacles and potential barriers to 
accomplishing their goal. 

5. The support of the design team and client are important. Sufficient aid from outside sources is 
imperative, including technical support staff and colleagues in the field. 



384 




References 

Bannan-Ritland, B. (1999). Teaching Instructional Design: An Action Learning Approach. Retrieved from 
University of Georgia, Instructional Technology ForumWeb site: 
http://it.coe.uga.edu/itforum/paper37/paper37.html 

Cannings, T.R., & Tally, S. (2001). Multimedia and Online Video Case Studies for Preservice Teacher 
Preparation. World Conference on Computers in Education, 699-710. 

Ertmer P.A., & Russell, J.D. (1995). Using case studies to enhance instructional design education. 
Educational Technology, 35(4), 23-31. 

Grabinger, R.S. (1996). Rich environments for active learning. In D.H. Jonassen (Ed.), Handbook of 
research for educational communications and technology. New York: Simon & Schuster 
Macmillan, 665-692. 

Hayden, M.A. & Ley, C.J. (1997). Collaborating with technology: Teaching a class on the world wide web. 

Journal of Family and Consumer Science, 89(2), 25-27. 

Julian, M.F., Larsen, Valerie A. & Kinzie, M.B. (1999). Compelling Case Experiences: Challenges for 
Emerging Instructional Designers, AECT Conference, 7-28. 

Kinzie, M.B, Hrabe, E. & Larsen, V. A. (1998). An Instructional Design Case Event: Exploring Issues in 
Professional Practice. Educational Technology, Research and Development, 46(1), 53-71. 
Kovalchick, A.M., Hrabe, E., Julian, M.F. & Kinzie, M.B.( 1999). ID case studies via the World Wide Web. 
In the ID casebook: Case studies in instructional design, P.A. Ertmer and J. Quinn (eds.), Merill, 
Prentice Hall, New Jersey, 141-148. 

Merseth, Katherine K. ( 1996). Cases, case methods, and the professional development of educators, (digest 
ED401272), ERIC Digest Number:95-5. 

Perry, G. (2000). Video case studies and teacher education: a new tool for preservice education. 
Unpublished manuscript, Pepperdine University, Culver City, CA. 



385 




Distance Learning and Role Play: A Web-Bard Pedagogy 

Mary Ann Kolloff 
Kevin Rahimzadeh 
Eastern Kentucky University 

As most any high school or college English teacher can verify, it is the rare group of students that 
approaches the study of Shakespeare with anything other than a mixture of anxiety over the difficulty of 
making sense of the plays’ language and annoyance that, once again, they will be forced to engage in an 
activity they find both unjustifiably difficult and irrelevant to their lives. Students dread the “Old English,” 
as they mistakenly call it, they will be expected to decipher and the daily plot quizzes they know they 
cannot pass (at least not without the help of SparkNotes.com). They balk at the maddening glossary at the 
bottom of the page, interrupting their reading every other line, and the confusing crowd of characters (all 
with funny -sounding names) they will be expected to disentangle. They worry over the lectures and 
discussions that will parade out all the usual half -understood English class shibboleths such as “motif,” 
“theme,” and “symbol.” Most of all, and worse still, they fear the imminent prospect of being bored out of 
their minds. 

The apprehensions students bring to Shakespeare are well understood by most teachers, the best of 
whom have at their disposal a ready reserve of tips and techniques, on call at a restless moment’s notice. 
More unexpectedly, surveys conducted with pre - and in -service teachers show that the worries Shakespeare 
provokes are not limited just to students. A good number of those undergoing teacher training and 
development in our classrooms, at both the undergraduate and graduate level, admit to frustrations similar 
to those expressed by students. Students, many have come to determine, are regrettably correct and 
Shakespeare, “really is” “stuffy,” “unapproachable,” “difficult” and, it must be said, “boring.” Still, they 
are expected to teach him and so they will buckle down, focus on the task at hand, muddle through as best 
they can — and look forward to “The Great Gatsby” in the spring. 

Background 

The project described here began as an effort both to learn more about student and teacher 
attitudes toward Shakespeare, and to try to influence those attitudes through a combination of online and in- 
class instructional techniques. Each semester for the last three years a faculty member from the 
Department of English has joined with a member of the Department of Curriculum and Instruction at a 
Southern -Central Regional University to engage in an interdisciplinary project that pairs synchronous 
online role play in the course management system. Blackboard, with traditional face-to-face meetings. The 
purpose of the project has been to explore Shakespeare’s life and works with pre -service and in-service 
teachers in a course taught in the University’s College of Education. While instruction in Shakespeare has 
been the project’s primary rationale, of nearly equal importance was the desire to model online role play as 
a promising teaching strategy for these same students, one that will move students and teachers alike 
beyond their initial, often ambivalent or even negative, assumptions about studying Shakespeare. 

Prior research has shown the efficacy of role play as an educational technique, which asks students 
to enter an imaginary world or to consider a problem or idea in light of a particular, pre -defined situation 
(Van Ments, 1989). Research also indicates that online role play is becoming an increasingly popular 
teaching method, one that is a logical Internet-era extension of traditional role play pedagogy (Bell, 2001a; 
2001b; Freeman & Capper, 1999). Role play, whether online or face-to-face, is useful both because it is 
highly experiential and because, as Bell has written, “it can lead to powerful behavioral and attitudinal 
outcomes” (2001a, p. 68). Perhaps most important, role play is fun; it is one of the few classroom 
activities that is enjoyed by nearly all students who engage in it (Van Ments, 1989; Bell, 2001a). As for 
instruction in Shakespeare, little work seems to have been done regarding electronic instruction in general, 
let alone online role play in particular (Birmingham, Davies, & Greiffenhagen, 2002). 

Methods 

Project Activity Summary 

The Shakespeare role play project as designed and implemented it contained four separate but 
interlocking activities, each building on the others over a span of several weeks. These activities were: 
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• Asynchronous Discussion Questions 

• Synchronous Online Interview with Shakespeare 

• Asynchronous Online Assessment 

• Face-to-Face Assessment with Shakespeare 

While the first activity, in which students formulated discussion questions for their interview with 
Shakespeare, worked well as an asynchronous online activity, it can also be undertaken successfully face- 
to-face. Likewise, student assessment of the interview would work equally well either electronically or in 
the classroom. A face-to-face meeting with the instructor role playing Shakespeare was, however, 
considered to be not just desirable but crucial, for reasons discussed in more detail below. 

Project Activity One: Asynchronous Discussion Questions 

Much work was done to prepare for the online role play. Students in the class began the project 
several weeks before the interview was scheduled by reading and researching material to formulate pre-role 
play questions for Shakespeare. These questions provided a useful index to students’ initial attitudes 
toward the study of Shakespeare; they also indicated what students think is most important for them to 
learn about him and what they think is most important to teach their own students. Student -generated 
questions for Shakespeare tended to break down into the following broad categories: 

• Biographical: (For example, “How many children did you have?” “When did you retire from the 
stage?” “How old were you when you married?”) 

• Historical/Cultural : (“Why were women not allowed on stage?” “Did your company ever 
perform at Court?” “What sort of people attended your plays?”) 

• Pedagogical: (“At what age would you introduce modern students to your plays?” “What are 
some ways to teach Macbeth?” “Should students be forced to memorize lines?”) 

• Personal: (“Did you love your wife?” “Were you jealous of Ben Jonson and Christopher 
Marlowe?” “Did you base characters on people you knew?” “How were you inspired to write Romeo and 
Juliet!”) 

An analysis of six semesters’ worth of these preliminary questions yielded the following breakdowns: 

• Biographical: 31 questions asked 

• Historical/Cultural: 71 questions asked 

• Pedagogical: 29 questions asked 

• Personal: 261 questions asked 

It was expected that students posed to Shakespeare twice as many personal questions as all others 
put together. Responses to the other categories of questions are readily available, after all, in reference 
books and on the Internet. Hence students took advantage of this sort of imaginative activity by asking 
questions they could not find answers to in traditional resources, questions that for one reason or another 
sparked their interest in Shakespeare. What ended up eventually frustrating so many students, however, 
was that personal questions, once they were submitted in the interview, were the very ones that were either 
ignored or flatly denied an answer. The instructors, naturally enough, did not wish to comment on matters 
they, or any scholar of the period, can know nothing about. This position was taken out of fairness to 
Shakespeare himself (asking a man if he loves his wife represents, after all, a considerable breach of good 
manners), and the role play instructors did not wish to spread literary gossip without being able to identify 
it as such, which, under the implied rules of the role play, they could not do without destroying the illusion 
that Shakespeare himself was online. While a good number of students expressed frustration, even anger, 
over refusals to discuss personal matters — “Why aren’t you answering me????” was a question that 
appeared many times on the screen — they were understanding when the reasons for those refusals were 
explained to them later during the face-to-face debriefing session. 

It can only be concluded that, whatever their initial anxieties about studying and teaching 
Shakespeare might have been, the sheer variety and thoughtfulness of the questions posed to Shakespeare 
in the weeks leading up to the online interview were clear indications of a genuine interest in the man, his 
works, and his times. The questions also showed a deeply felt need on the part of these teachers to learn 
how best to approach Shakespeare with their own students, and they confirmed that for most of them, some 
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sort of biographical or personal approach seems to be the best choice insofar as they recognize that this 
approach has done much to spur their own interest. 

Project Activity Two: Synchronous Online Interview with Shakespeare 

The online role play was the centerpiece of the entire project, and occurred when the role playing 
instructor logged onto the online environment as “William Shakespeare,” thus concealing his personal 
identity and allowing students to imaginatively conduct an hour-long interview with Shakespeare himself 
(Ko & Rossen, 2004). Only after the role play activity, during a face-to-face assessment session with the 
class, was the identity of “William Shakespeare,” a professor from the university’s Department of English 
and Theatre, revealed. After a few moments exchanging awkward hellos, he began by asking the class 
what they most would like to know of Shakespeare. Many of the questions posed during the remainder of 
the chat session were the same as those offered in advance through the discussion forum. Interviews 
inevitably, however, took on a life of their own as students grew comfortable with the online activity and, 
as they stated later, began to buy in imaginatively to the idea that they were speaking to Shakespeare 
himself. In choosing which questions to respond to, the role playing instructor had, of course, his own 
hobbyhorses, ones that he thought would benefit students in their own classrooms. Questions, for example, 
on Early Modern culture were given high priority, the answers to which may not be easily accessible in a 
print or online source. Gender roles in Shakespeare’s era and religious beliefs in the time period and how 
they might have affected the plays, were topics that were almost always taken up. Questions about 
individual plays were also answered, as the instructor did his best to keep the focus on various ways to 
teach them. Finally, historical concerns such as the publishing or staging practices of the era were almost 
always responded to. While the instructor made no effort to speak in blank verse, he strove for a formal 
tone, paying as much attention as possible to spelling and grammar before hitting the Enter key; because he 
did so, the tone of student discourse appeared to rise, in that questions and responses seemed to become 
more thoughtful and carefully formulated as the interview progressed. 

Perhaps the most interesting aspect of the interview activity took place in the half hour or so after 
Shakespeare left the virtual classroom and students continued to post questions, now to one another, and 
offer reflective comments. While much of this discussion comprised of complaints over questions that 
were not responded to or answers students did not much like (many students took umbrage, for instance, at 
the notion that Shakespeare borrowed the vast majority of his plots, or that he most likely intended to earn a 
good bit of money in his chosen career), much of it centered on the exercise itself and its usefulness to them 
as both students and teachers. Students also raised important epistemological issues, asking one another 
how they are to know whether Shakespeare’s comments were accurate, and whether they had been 
“seduced” into believing in Shakespeare’s truthfulness simply because his name kept popping up on their 
screens. As Van Ments (1989) pointed out, the problem of ensuring accuracy within a fundamentally 
imaginary scenario is inescapable in role play instruction (p. 28), and it is clear from observations of this 
phase of the activity that this was the case with the Shakespeare project as well. But in the end what might 
have been a considerable obstacle to learning -the aura of uncertainty that grew around Shakespeare’s 
statements once he left the chatroom-became a clear advantage once students began to address the problem 
directly. In fact, what was most rewarding about this post-interview discussion was the way students were 
observed exercising significant critical thinking skills as they evaluated the interview, assessed the 
accuracy and usefulness of Shakespeare’s statements, recognized their own biases and presuppositions 
about studying Shakespeare, and drew disparate conclusions about the relevance of the exercise (Khan, 
1997). 

Project Activity Three: Asynchronous Online Assessment 

The students’ task over the following week was to complete electronic surveys that allowed them 
to clarify further their thoughts on the significance and usefulness of the role play session. The survey 
consisted of the following questions: 

1. What expectations did you have going into the role play interview? 

2. Did anything surprise you about the role play experience? 

3. What did you find useful about the experience? 

4. Did Shakespeare’s statements strike you as accurate? How might you verify the accuracy of his 
statements? 

5. Evaluate your experience with the role play in terms of your prior experience with Shakespeare. 
Did it add to your knowledge of Shakespeare? 
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6. Did the role play increase your motivation to want to learn more about Shakespeare and his 
works? 

7. Have you changed your opinion about the appropriate age to introduce students to Shakespeare 
based upon the role play? 

This survey has proven to be a highly effective assessment tool; in particular, it provided 
insightful student -generated answers to the larger question surrounding this project: What was the point of 
the Shakespeare role play? The surveys suggested that students understood the project as attempting the 
following: 

• Teach facts about Shakespeare and his era. 

• Provide insights into his plays. 

• Model a technique that would work in these teachers’ own classrooms. 

• Provide fun. 

• Motivate teachers to want to learn about Shakespeare. 

• Motivate teachers to want to teach Shakespeare, even when his works remain, as they do for the 

middle school teachers, outside the established curriculum. 

Below are examples of typical student comments regarding each of these purposes: 

1. Teach facts about Shakespeare and his era. 

1 learned more about Shakespeare in that chat than 1 ever learned from studying him in high school or 
college. 

I learned so many facts I did not know. I think it was useful because Shakespeare talked just like people 
today do, and he did a lot to get us interested in his life and times. 

2. Provide insights into his plays. 

I found out lots of information about his plays that I would never have known unless I had done this chat 
with Will; his plays are things I usually try to avoid because they are too difficult. 

I liked his explanations of the plots of the plays and their overall themes, especially about Hamlet and King 
Lear. I’m not very familiar with these plays, so I found the discussion of these plays enlightening. 

3. Model a technique that would work in these teachers’ own classrooms. 

Before the role play I thought Shakespeare was confusing, but now I see a way that you can incorporate 
him in ways other than just reading his plays. Role play can get students involved just like we were, and 1 
would like to do a chat session like we did. 

It is a great idea to use in our own classrooms! It will allow students to escape for just a little while and 
try to put themselves in Shakespeare ’s times. 

4. Provide fun. 

Role play made Shakespeare a REAL person! I loved the fact that I felt like 1 was talking to HIM! 
It was a ton of fun. 

1 loved it! 1 give it a 9 out of 10 because it was such fun. I don ’t give it a 10 because it wasn 't long enough 
and I didn 't get all my questions answered.® 

5. Motivate teachers to want to learn about Shakespeare. 

After the role play, 1 wanted to examine his life more closely. I would also like to find out more 
about his career as an actor and writer. 
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I found role play valuable because l saw Shakespeare in a whole new light ! It was like he was 
actually there, talking , discussing and explaining his life and works. This was very useful because I have 
become more interested in William Shakespeare and am eager to read more of his plays and sonnets. 

6. Motivate teachers to want to teach Shakespeare. 

The role play made me see that Shakespeare could be introduced and understood at an earlier age. 

I very much appreciate Shakespeare now because I have learned more about the period. So I want 
students to learn about him too. I think the life of William Shakespeare is interesting and his pieces of work 
astonishing. 



Project Activity Four: Face- to -Face Assessment with Shakespeare 

A week or two after the role play session, the role play instructor met with the students in person 
to discuss and evaluate the exercise. Research into role play emphasizes the importance of a debriefing 
session of some type (Bell, 2001a), and as Van Ments (1989) has written, debriefing is an indispensable 
“two-way process,” one that “establishes the learning in the student’s mind” (p. 49). Odd as it may seem, 
students appeared a bit startled when a rather ordinary looking college professor walked into the classroom 
rather than the Bard himself. Many pointed out during this debriefing session that the identity of 
Shakespeare had been a subject of intense speculation in and outside class in the weeks leading up to the 
face-to-face meeting. This meeting provided another opportunity to assess all stages of the exercise, but its 
most important function was that of offering further significant points of instruction, which it is 
recommended be conducted in class if students are to evaluate the online activities with suitable distance 
and objectivity. For what students most seemed to need at this stage in the project was a sustained 
examination of the benefits, drawbacks, and epistemological difficulties online discourse and role play 
present. Moreover, they required both a firmer sense of the biographical uncertainty surrounding 
Shakespeare and, more generally, a more complicated perspective on the limitations surrounding any effort 
at historical and biographical reconstruction. Relative to other playwrights of his era, quite a bit about 
William Shakespeare is known. But relative to what modern readers and theatre-goers “would like” to 
know about him, very little is known indeed. Once it was explained why all their personal questions about 
Shakespeare lingered on the screen, unanswered, or why one can say with certainty that Shakespeare acted 
at the Globe in the first decade of the Seventeenth Century, but one cannot explain with any certainty at all 
as to what might have compelled him to write “Othello ”, students were left with a richer sense of historical, 
biographical, and literary complexity. These teachers seemed to appreciate these points. They frequently 
stated during this assessment meeting that the textbooks they use or will use in the classroom and the 
resources they consult to prepare for class leave little room for ambiguity, or for the sort of problematizing 
of settled assumptions the role play project was designed to effect. 

Conclusion 

Role playing Shakespeare is doubtless a promising way to teach and motivate students, and if 
students are to be taken at their word, then online role play might also prove an equally successful 
instructional technique in these teachers’ own classrooms. Virtual role play appears to allow students to 
make necessary imaginative leaps to engage a Shakespeare character without the emotion of 
embarrassment over something “too” realistic — a walking, talking, yellow tights -wearing Shakespeare - 
hindering those leaps In online surveys completed after the interview, students commented again and 
again on the surprising “reality” of the role play activity. In fact, the term “real” was used more frequently 
than any other as an overall description of the experience: 

I was amazed by how quickly you could become wrapped up in the role playing. It was very easy 
to let yourself believe that you were really talking to William Shakespeare. 

Will seemed so real , even though a part of me knew it wasn 7 REALLY Shakespeare. 
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The role play made Shakespeare seem so real to me; the chat session could help students understand that 
he was a real man. 

I was shocked that the person knew so much about Shakespeare 's life and works. I practically thought for 
a while it was really Shakespeare himself. 

To be honest, at first l thought role play was kind of dumb since Shakespeare is dead, but as he started 
talking, he came to life, and l really thought I was talking to him. 

The project’s realistic but not “too” realistic nature also explains, perhaps, what made it so much 
fun. The face-to-face assessment with students indicates that students found the interview with 
Shakespeare “real” enough to prompt an enjoyable imaginative response to Shakespeare, but not quite so 
“real” that the students’ attention was drawn too unduly to the discrepancy between what they perceived 
(that a man going by the name of William Shakespeare was conversing with them) and what they knew 
(that Shakespeare has been dead for four hundred years). 

In the end, it seems clear that what role play did unusually well was to satisfy the students’ 
longing— a longing they no doubt share with anybody who reads and enjoys imaginative literature -for 
authentic authorial presence. Asking students to consider what an author might have intended can be an 
illuminating approach to literature, and is surely a legitimate area of literary inquiry. Still, as in any 
consideration of authorial motive, the proper watchword for role play instructors seems to be this: Be 
careful, and while being careful, be honest with students as to why such care is necessary. So long as 
instructors make clear that they are aware of the difficulties involved in invoking authorial presence so 
dramatically, and share and discuss those difficulties in a direct and probing way, then role play of the sort 
presented here can be an appropriate and productive teaching tool. 
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Abstract 

The Shareable Content Object Reference Model (SCORM) aims to standardize electronic course 
content, its packaging and delivery. Instructional designers and e-learning material producer 
organizations accept SCORM’ s significant impact on instructional design/delivery process, however not 
much known about how such standards will be implemented to this process. This paper presents a project 
at an educational software production company that has decided to change its production line to conform 
to the SCORM standards. The paper provides the technical details of the project such as technical platform 
and the infrastructure for related components. The technical issues and decisions of asset management 
system, SCO development and repository will also be mentioned. Secondly, administrative issues of this 
process will be discussed. Administrative issues can be summarized as the changes in the work practices of 
the instructional design/development process, organizational resistance, and new roles and responsibilities 
of people in the standards compliant instructional design/development process. 

Introduction 

Today many universities and organizations offer courses through online learning systems. 
Especially, e-learning or Internet-based distance learning have gained significant importance and it is the 
most rapidly growing aspect of education and training in the world (Cagiltay, Graham, Lim, Craner, & 
Duffy, 2002; Lytras, Pouloudi & Poulymenakou, 2002). According to Murray (2000), “In the future, 
education will increasingly utilize distributed learning technologies” (p. l).While he and other researchers 
argue that there is promising future in the use of personal computers and the Web in education, there is no 
satisfying answer for the question of how these technologies can best be used to develop instructional 
materials in an efficient and effective way. 

Educational materials for online learning environments generally include a computer interface, 
several different data formats (e.g. text, graphics, image, voice and movie), an evaluation system to assess 
students’ progress, and several other support tools to support the learning environment (Robson, 2000). As 
stated above, preparing online educational systems has several unknown issues. An important issue is “how 
to produce and deliver quality content for online learning experiences” (p.l) (Robson, 2000). Generally the 
volumes of the materials that are prepared for online learning environments are very high. This makes it so 
hard to keep the content up to date and most of the times it is hard to update the content regularly. 
Formatting these materials in several different ways for different purposes is another difficulty. There are 
always portability and compatibility problems among online educational systems. 

Actually similar problems were also faced in software development field. Until early 80s creating 
new information systems were very clumsy work. For every new software project, programmers had to 
write a new code. Software developers’ solution to this problem was using object-oriented approaches. 

This brought modularity and speed to the software development process. A similar approach called 
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“reusable learning objects (RLO)” is becoming very popular among instructional designers of on-line 
systems. 

As stated by Robson (2000), reusable -learning objects represent an alternative approach to content 
development. In this approach the content is divided in small chunks. Each chunk is called as a “Learning 
Object” (LO) and each LO has a special educational role in the system (Robson, 2000: p. 1). Basically, the 
concept of learning objects helps instructional material developers to divide these materials in small chunks 
such a way that they can be reused for different instructional purposes. 

As stated above. Learning Objects are grounded in the object-oriented paradigm of computer 
science (Wiley, 2000). According to Wiley (2000), there will be a major change in the way educational 
materials designed, developed, and delivered in the near future. According to him, “learning objects” leads 
other approaches (Wiley, 2000) for implementing such a change. So, compliance with Learning Objects 
(LO) or Reusable Learning Objects (RLO) standards is a big challenge for the educational institutions. 

An important issue is the standards for defining the LO. For example IEEE Learning Object 
Metadata Working Group is working on some metadata standards for defining a learning object (IEEE, 
2002). Other e-learning standards body the IMS Global Learning Consortium “develops and promotes the 
adoption of open technical specifications for interoperable learning technology” (IMS, 2004, p.l). The 
Advanced Distributed Learning (ADL) Initiative adapts these and other organizations’ collection of 
specifications to form a new high level standard which is the Sharable Content Object Reference Model 
(SCORM). The SCORM defines “the interrelationships of course components, data models and protocols 
so that learning content objects are sharable across systems that conform with the same model” (ADL, 

2004, p. 6). 



Purpose of the Study 

The SCORM promises many opportunities for those who develop large scale educational e- 
materials. However, there are many unknowns about the implementation of SCORM specifications and 
guidelines to real life instructional design/development process. These unknowns can be grouped under 
two main headings. One of them is technical issues and the other one is administrative ones. In this study 
we will report and share our experiences of how the changes to SCORM compliant content production 
process planned, developed and implemented at an educational software company. 

The Organization and the Project 

The company has been providing software products, services and creative multimedia content on 
cross platforms including internet, mobile, digital TV and electronic media since its foundation in 2001. As 
results of the undertaken projects and works done, the company has managed to produce over 400,000 
digital objects during the 2 years period. Consequently, the company staff encountered problems of 
archiving, retrieving and re-using existing objects as well as newly created objects. So, the company has 
decided to establish and re-engineer the production process and digital objects archive. A pioneering team 
was established and the initial steps were taken. The team proposed ADL SCORM as a base of the project 
and the proposal was accepted by the company board. Then a dedicated project team was established. The 
team consists of one systems analyst, one software engineer, one instructional technologist, and one library 
and information specialist as full time staff members. The project entitled ''Standard Learning Objects 
Creation, Searching and Publishing Platform" and received support grant from Technology Forecasting and 
Assessment Directorate/ The Scientific and Technical Research Council of Turkey. 

For the project, “reusability” was a critical word and it should be well defined in order to meet the 
expectations. The company aimed to keep its dynamic and productive skill in a cost-benefit way. For this 
purpose, reusability is understood in two ways. One is using the assets previously produced in new projects 
as they are. Other understanding is using the assets previously produced while producing similar assets to 
shorten the production time and effort. Identifying the assets clearly and searching them easily are critical 
functions. 

The users of this system are asset producers (Graphic Designers, multi- media developers, etc.), 
teachers or designers seeking learning objects to use in their work, e-content developers, and managers of 
asset production units. These users interact with three basic modules which are “Defining RLOs and 
SCOs”, “Searching RLOs and SCOs” and “Publishing the output”. These users, the modules and their 
relations are showed in a structured way in Figure-1. Users are showed as User-I, User-II and User-Ill since 
they can represent different people from different disciplines. 
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General Issues for Administrative and Technical Challenges 

The aims and expectations described above were critical for designing the platform. Different 
people from different disciplines, who were usually working in different units, should have roles in content 
production. In such environments effective communication is important and having a common language 
within these subgroups is critical. Technical platform used in such environments can help setting a common 
way of working and forming a common language. 

This project was seen as creating an online-archiving environment which is conformant to 
SCORM standard and it is clearly set that some cultural changes in the company should be expected and 
promoted in this period. For the administrative side, the procedures and workflow is criticized and 
redesigned according to learning objects approach. And a new role is defined in this workflow. This role is 
positioned as a RLO -Librarian for controlling and entering the metadata of the learning objects. 

The technical platform is also designed with the idea of changing some parts of workflow which 
people are already familiar. First it is thought that learning objects should be stored in a common place 
regarding the projects that are used. So handling a project and handling the RLO Repository are different 
from each other. For staff, who used to think just as working in a project, this mechanism is completely 
new. Common repository concept is required to be accepted and supported by the staff. It is expected by 
the administration that internalizing such a concept will affect the production style of staff. It is hoped that 
the frequency of the number of reusable assets produced will be higher as this approach is supported. 

Technical Issues 

Technical Platform 

Usually, content production tools are used for packaging and generating required files of SCORM 
standard. The outputs of these tools are usually files which are stored in the hard disks. In the company, it is 
observed that asset production occurs in producers’ personal computers and after completing their 
production, their products are carried to the related project folders. However, after a while in such a system, 
only the people who worked in those projects remember their previous works and only they can reuse them, 
or it takes time to search many projects for similar assets. As the number of projects increase, this 
nonstandard way of production becomes a cultural characteristic of the company or production units. 

To help this change, support of technical platform on the workflow and production style, and a 
change in technical procedures is considered as critical. It is decided to store only the resource files of 
approved assets in a common place. In addition, metadata of these assets and some relevant information are 
decided to be stored in the database. By this design it is believed that the learning objects will be searched 
in the database and new packages will be generated easily, whenever it is needed. 

The production process takes time and includes different people. This means there are different 
states of RLO during the production cycle. The new RLO -Librarian role has responsibilities of controlling 
the whole process, entering relevant metadata, controlling the metadata entry of different people, and 
approving/regretting the production. Approving is important for protection of the repository from 
junk/meaningless objects. For this reason, the working environment is determined as three parts: Local, 
temporary and actual RLO repositories (Figure -2). Use of temporary and actual RLO Repositories is shown 
in the asset production process (Figure -3). 

Components of the Platform 

There are four sub-modules of the platform which are used for defining the metadata of assets and 
SCOs, for searching RLOs over metadata defined, for packaging according to SCORM and for publishing 
the packaged content. Publishing module seems unrelated to content production concept, but since the 
clients may not have SCORM compliant tools, it is added to the project in order to facilitate the use of 
RLOs. For SCORM compliant content production, availability of SCORM compliant content publishing 
tools may become a challenge. 

Another challenge had occurred during the development was the sequence of implementation. One 
way is considering the packaging module first and handling metadata definition within this module. The 
other way is considering definition module independent from packaging module. Although, the first way 
seems more practical, the second way is selected since, focus of the administration is on the control of the 
asset production and handling the repository is considered independent from handling the projects. 
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Procedural Components 

Parallel to technical development, the company believed that the procedures and change in the 
production cycles or styles are critical. Since this is a system and RLO is a way of thinking, all related 
components should be consistent with the approach. With criticizing the previous experiences and 
organizational structure of the company, “Asset Production Process” (Figure -3), “SCO Production Process” 
(Figure -4) and “SCO Evaluation Process” (Figure -5) are defined for new projects. The company decided to 
convert the previously developed material to SCORM compliant versions. In addition to the procedures 
above, “Migration to SCORM compliant process” (Figure -6) and some additional forms (Figure -7) are 
defined. 

Other Technical Issues and Experiences 

While producing the assets, the producers are expected to fill out some parts of metadata of the 
asset they work on. Some times these assets are very small and they can be produced in a very short time. 
For SCORM standard, there are more than 70 tags. Some of them should be defined by clearly thinking like 
key words. When the number of tags is high, to facilitate the efficient use of the system, the time and effort 
of the producers should not be so much. Decreasing the effort and time of data entry is one of the major 
aims of the platform. To achieve this aim, some templates are used for automatic filling, and some 
shortcuts are generated. 

For countries whose language is not English, translation of metadata tags and the need for studies 
about concepts are challenging aspects that should be considered before starting to such project. During this 
project, the company tried to translate the metadata information and their explanations for the user 
interface. However, it was seen that one-to-one translation may not be satisfactory. Some conceptual 
changes are required. This is critical especially to decrease the time of entering metadata information to the 
database. For example, when a 2D animator uses the system to enter a new image, the labels and the 
explanations of the metadata should be easily understood. 

A conceptual study about the predefined terms is needed in order to explain them clearly and to 
have a common language. These are especially necessary for terms or words which can have different 
meanings for different groups. For example, the meaning of high or low can change according to 
perception. Such terms should be clearly defined for each of the groups. And, they should have common 
meanings for all. Usually it is easier to have a common language when these people are from the same 
discipline. Use of same platform, filling out same fields with different views will probably cause problems 
especially when the number of assets recorded becomes very high. 

In addition to conceptual studies, additional standards can be used within SCORM where it is not 
clearly set. In this company, these places are “keyword” and “classification” tags of the metadata set. Since 
the system can be considered as an archiving system, searching should be very effective. Again, if different 
people define different key words with same or similar meanings, search results will be less reliable. While 
filling these fields, it will be beneficial to use some internationally accepted standards like “Thesaurus of 
ERIC Descriptors” or “Library of Congress Subject Headings (LCSH)”. For the classification field, 

“Dewey Decimal Classification System” which is a well known system in school and public libraries. 

Using such standards is a challenging point that very little examples are found in the literature to compare 
and discuss. 

Administrative Issues 

Changes in Work Practices 

Learning Objects requires a different approach from previous work approaches. For the 
management side, this change is very valuable since cost of such material is very high, and reusing the 
learning objects motivates them. For instructional technologists and instructional designers, the way of 
thinking becomes consistent with the learning objects approach. The way the asset producers use to work is 
expected to change. They began to transfer some parts of the metadata, their knowledge and ideas about 
their products through the platform. Finishing a task means not just drawing for a graphic designer, but 
entering its metadata which he/she could know and is expected to fill. 

As the number of RLOs increase and as the benefits of the system is accepted by the users, 
probably the first attempt to produce an asset will be searching. It was observed that because of the 
communication problems in a crowded team, sometimes similar or already same assets are produced by 
different people at different times. 
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Another expectation from changes can be development of new viewpoints of designing or 
producing more reusable assets. Here reusability can be re-using as it is or re-using it as a base. Yet, it was 
seen that asset -producers could prefer reproducing instead of making revisions. 

Organizational Resistance 

Organizational resistance was an expected issue in this project. So, from the beginning the change 
should be considered for everyone affected. Aiming to be SCORM conformant is not enough, it is critical 
to answer the how questions. Because it seems that answering these questions will give hints about how the 
users, designers, developers and managers will change. Therefore, probably in every aspect there will be 
some resistance. 

First of all the resistance expected from the asset producers, because this change increases their 
responsibilities. It is observed that they were unwilling for using this system, especially when they have 
noticed the number of tags expected to be entered. Their argument was they cannot finish neither their 
production nor entering these data, and due dates of the projects will be affected. In order to overcome this 
resistance, project team paid more attention to usability. Some additional mechanisms are added to the 
system. For example, when a new project is started, a new template can be generated in order to fill some 
fields by default values. So, users only enter necessary items or make revisions. In addition to templates, if 
some fields can be filled due to another field, those automatic operations are developed in the platform. For 
example, when an image is selected, some properties which can be gathered from the system (e.g. file 
extension, type, file size, etc) are filled to related tags. 

Another solution to improve usability is separating the related fields and unrelated fields according 
to the users’ role. While trying to determine the relationship between the fields and the roles it is seen that 
the usability efforts are very meaningful to overcome the resistance. Figure -8 shows the list of 
“Translations & Explanations of Fields” which is prepared for reference purposes. Figure -9 shows the first 
worksheet which is prepared to be used while interviewing with the team-leaders of different production 
units. When the interviews began, they showed resistance to the idea and mentioned that they cannot fill all 
that fields and this system will be blocked due to their work- load. Observing this resistance, another 
version of worksheet is prepared (Figure-10). This time, within a column, it is given that the field was 
automatically filled or who’s (as role) responsible for that field. New format of the worksheet was very 
effective. The results show that nearly all fields are considered as needed although they can be optional. 
These results are consolidated according to roles (Figure -11) 

New roles and responsibilities 

At the beginning of the project a new librarian-like position was suggested to the management. 
Until that time, when there was something about content, just instructional technologists and content 
experts were seen as related people. Since the number of RLOs is expected to be very high in the future, it 
is believed that tracking and classification of those objects requires the know-how of librarians. The 
responsibilities of this role are entering some fields, ensuring the required fields are filled correctly, and by 
approving uploading the end-products to the actual repository. Moreover, since a learning object is 
generated to be used in somewhere, the uploaded and approved learning objects are stored as they were. 
When it is needed some versions are kept with the determined methods. A rule was set that “updating an 
approved learning object is possible just for bug fixing”. By this way the local copies which are found in 
developers’ computers are the ones that they are working on and the ones which can be neglected. 

Summary 

In this paper, experiences of a company trying to move toward SCORM compliant content 
production are explained. It is seen that, administrative issues are also critical as technical issues for such a 
period. Moving toward a SCORM compliant content production can be seen as forming a common 
language between different subgroups in the company. Changes in the production style and workflow can 
cause resistance and should carefully be considered. To overcome these resistances there may be many 
topics to go on like conceptual studies, additional standards, usability factors. 

For the technical side, the understanding of “reusability” and expectations of the system is critical 
and should be carefully discussed at the beginning of such a movement. Reusability is understood in more 
than one way in the company. One is reusing the assets produced as they were; the other is reusing the 
assets produced as a base while producing new assets. These expectations cause the solution to include an 
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archiving system in which storing a reusable learning object means different entries of different users in a 
workflow. As a result use of common storage mechanism as approved and temporary bring additional 
constraints while designing such a system. 
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Figures and Graphics 



Figure-1: General Structure 1 



Figure-2: General Structure 2 




Figure-3: Asset Production Process 



398 







Figure -4: SCO Production Process 



Figure -5: SCO Evaluation Process 
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Figure-7: SCO Form 
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Figure-8: Translations & Explanations of Fields 



Figure-9: First Worksheet about Fields 
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Figure- 11: Results of different groups’ acceptance 
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Exploring the Potential of WAP Technology in Online Discussion 

Chwee Beng Lee 
University of Missouri-Columbia 

Introduction 

The intent of this study is to explore how WAP (wireless application protocol) technology 
mediates online discussions. The key focuses of this research are on the implications of WAP technology 
for online discussions, the types of discussion topics that are most suitable for WAP-based discussions and 
the finding of the combination of WAP- and WEB-based discussions. 

Multiple methods of inquiry were employed in this study. A survey, face-to-face and focus group 
interviews were conducted to find out the participants’ perceptions of using the WAP-enabled mobile 
phone as a communication tool, and how such a tool has facilitated or impeded their learning processes 
both technically and cognitively. A content analysis was made on the postings generated from WAP- and 
WEB-based forums. 

This study is one of the very first attempts at investigating the use of WAP-enabled mobile phones 
for online discussions in one of the higher education institutions in Singapore. By exploring the potential 
of this newly developed technology, it is hoped that this study may provide guidelines to educators as they 
reflect on the way online discussions can be integrated into a course and also laying a foundation for future 
development of a mobile online learning environment. 

Online Discussion as a Sociocultural Tool 

In this study, participants were requested to participate in asynchronous online discussions. 
Asynchronous communication refers to anytime -anywhere communication between two or more 
individuals. In such a communication, participants are unrestrained by space, time and pace. They may 
read the messages that are posted in a central location or delivered to their email box at their own 
convenience. Wireless technologies can free learners from the need to be tied to a particular hard -wired 
location to access information (Gunawardena & Msisaac, 2004). There are several common forms of 
asynchronous communications. These include e-mailing, threaded discussion and newsgroup. In this 
paper, online discussions refer to the use of a web-based application that enables participants to create and 
edit messages that are stored in an area that is accessible to group members who organize messages into 
“threads” of conversation (Curtis & Lawson, 2001). In this study, the participants were given the option to 
participate either in the WAP- or WEB-based forums via WAP-enabled mobile phones or their own 
personal computers. 

Asynchronous online discussion offers a range of advantages to learners such as self-paced 
participation and reflective thinking. In an online discussion environment, students are likely to obtain 
more experiences managing their interactions with the content themselves. Online discussions require 
learners to manage their own learning, free from the teacher-centered settings; this then provides an 
opportunity for learners to progress independently (Lee, 2002). Researchers also found that online 
discussions are more task-oriented than face-to-face discussions. Also, reading and writing are employed 
discursively as a means of focusing members of a classroom community on matters of joint interest 
(Lapadat, 2000). Moreover, as asynchronous discussion is text -based, the meaning of a text -only message 
is divorced from the sender’s physical presence and verbal delivery style and all that remains is what the 
person actually says (Altanus, 1997). 

This study does not intend to make comparison between different technologies on how they 
perform in online discussions. Rather, it seeks to explore how the various technologies compliment each 
other and mediate online discussions. When engage in online discussions, learners should be given more 
avenues to log on to participate. Web-based course when combined with other CMC tools such as email or 
bulletin board allows the learners to learn and follow their own path, enriching the exchange of ideas 
among learners (Box ,1999). 

One of the key concepts in Vygotsky’s work is the zone of proximal development (ZPD). The 
ZPD is the range of difference between what an individual may accomplish in an activity or task alone and 
what he or she may accomplish in the company of others (Althauser & Matuga, 1998). It is in the ZPD that 
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scaffolded learning takes place to support learning. Scaffold assistance can come from both cultural tools 
and more knowledgeable peers or experts in one’s learning environment (Jarvela & Hakkinen, 2000). 

Sociocultural researchers point out that instruction should take place in an environment in which 
learners use socially mediated and intellectual tools to achieve cognitive development (Rogoff 1990; 
Salomon 1993). Bonk and Cunningham (1998) comments that collaborative technologies can offer 
opportunities for both peer and mentor electronic guidance and feedback that stimulate student discussions 
and internal reflections in a scaffold learning activity. It is through online discussion that they may “voice 
their opinions and reflect on their learning, thereby increasing inter-psychological and intra-psychological 
activities to promote individual’s cognitive development’’ (Zhu 1998, p.234). Online discussion is a key 
mediational tool for “external display of students thinking processes and interchanges’’ (Jarvela and 
Hakkinen 2000, p.8). It promotes “exciting online learning communities’’ (King 1998, p.368). 

Discussions on brainstorming and case studies were adopted in this study based on the rationale 
that both types of discussion could enrich the online discussions by offering students opportunities to 
engage in higher levels of thinking such as critical thinking. Chong (1998) reported in her study that 
participants claimed that they have learned tremendously from the magazine articles and that authentic 
cases taken from real-life provided the complexity necessary to encourage critical thinking and logical 
argumentation. Authenticity in case studies did provide a challenging and real life learning environment. 
Also, empirical investigations have found that electronic brainstorming groups have generated more ideas 
than verbally brainstorming groups, particularly for larger group sizes (Gallupe, Bastianutti & Cooper, 
1991; Gallupe, Dennis, Cooper, Valacich, Nunamaker & Bastianutti, 1992). According to the study carried 
out by Dennis (1993) and colleagues, electronic communication among members has improved the idea- 
generation performance of large groups. 



Methods of Inquiry 

The participants in this study were the pre -service teachers who took the course ‘Instructional 
Technology.’ In this course, students learn how to effectively integrate IT into their classroom practices. 

To create a constructivist learning environment, the institution adopted Blackboard, an online learning 
delivery and management system that allows students to learn independently and instructors to customize 
the e-learning packages according to their students’ need. 

This research program was embarked in 2001. A class of twenty pre -service teachers, each with 
certain characteristics that might represent the target population was selected to take part in this research 
program. On the other hand, as they had a common teaching subject which was Chemistry that would 
make the discussions more subject-focused. All the participants were given WAP-enabled mobile 
handsets for their participation in the online discussions. They could access the online threaded discussion 
forum using the given URL anytime and anywhere via their WAP-enabled phones or the web-based forum 
via home computers . The participants were required to take part in all the 6 forums which consisted of 3 
case studies (with one as a pilot test) and 2 brainstorming discussions. 

A pilot test was carried out to test the application and also to explore the implications of the 
discussions generated within that week. During this pilot test, pre -service teacher and their course 
instructor participated in the case study online discussions . The information gathered and the preliminary 
analysis was of great value as it helped to identify some possible obstacles and helped to refine the design 
of the application. Also, it helped to refine the structure and content of the discussion forums. 

A survey for the pre -service teachers was conducted at the end of all 6 forums in order to obtain 
statistical evidences and also to generate deeper understanding of the study. A focus group interview with 
the pre -service teachers and a face-to-face interview with the class instructor were also conducted to gain 
an in-depth to the phenomena surfaced from the study; the transcripts can also be used to check the 
accuracy of interview records. The messages posted on the WAP- and WEB-based forums collected in the 
form of spreadsheet were then translated and analyzed into simple message maps that showed the flow of 
messages. 

To analyze WAP- and WEB-based discussions, Jonassen’s (2000) rubrics for quality discussions 
and Jarvela and Hakkinen’ s (2000) classification of such discussions were adopted and modified in this 
study. The messages posted in the both forums were categorized into five different types of discussions 
(suggestion, comment, elaboration, information-seeking and information-sharing) under four criteria with 
three levels of participation (high-level, progressive-level and low-level). These four criteria included 
accuracy, relevance, coherence of the messages, and the levels of perspective taking of messages. Postings 
that were irrelevant to the discussion topics were considered as redundant messages and were not analyzed. 
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Findings 

The result of the survey revealed that 65% of the pre-service teachers thought that brainstorming 
questions were more suitable for WAP-based discussions. The pre -service teachers felt that brainstorming 
questions would generate more new ideas, and since short messages were needed, it would be easier to 
share ideas in a more efficient way. The other 35% thought that case study questions were more suitable 
for WAP-based discussions because they were more focused than brainstorming questions. The results 
suggested that majority of the pre -service teachers thought that brainstorming questions were more suitable 
for WAP-based discussions. However, the findings from the messages posted on the forums and the 
transcripts from the interviews revealed another set of findings. 

The percentages of messages in both forums were low. Quantitatively, this suggests that 
regardless of whether it was a brainstorming discussion or a case study discussion, it does not affect the 
percentage of WAP-based messages posted in these forums. This outcome suggested that the nature of 
brainstorming discussions and case study discussions might not determine the discussion outcome. The 
discussion questions might throw some light on this phenomenon. During the focus group and face-to-face 
interviews, the pre -service teachers and instructor mentioned that the topics and the way the questions were 
phrased did determine the pre -service teachers’ responses. One pre -service teacher mentioned : “I think it’s 
the discussion topic that matters. For the discussion on school experience. . .1 have experience to share with 
them (the other students) so I would create a thread, if not, I would just see what people have to say.” 
Similarly, another pre -service teacher said: ‘‘for the school experience, I know the kind of environment so I 
can comment in that forum.” Their class instructor also emphasized that ‘‘the factor that determines the 
participation is not so much of whether it is case study or brainstorming, rather the nature of the discussion 
question itself. If the nature of the question can appeal to them, it will generate greater interest and thus 
they will participate more.” This might explain why some pre -service teachers preferred brainstorming 
discussions while others found that case study discussions more challenging. The more the students could 
relate the discussion topic to their own personal experiences, the higher the level of interest they would 
have and naturally the level of participation would increase. 

As shown in table 1, both WAP-and WEB -based forums produced limited number of high-level 
quality messages. However, WAP-based forums had yielded more low-level quality messages and less 
progressive-level quality messages than WEB -based forums. The physical and technical constraints of 
WAP-enabled phones might not have supported lengthy messages. Although statistical results suggested 
that WAP-based forums have produced more low-level quality messages than WEB -based forums, the 
usefulness and the contributions of WAP technology should not be ignored. In the survey that was 
conducted at the end of the course, 65% of the pre -service teachers agreed that WAP technology has helped 
to build a learning community. They believed that the WAP-based forum has formed a closely knitted 
group, and everyone was able to participate and learn from each other. 



Quality messages 
^Types^ffom 


High-level quality 
messages 


Progressive -level 
quality messages 


Low-level quality 
messages 


WAP -based forum 


6% 


29% 


56% 


WEB -based forum 


7% 


63% 


29% 



Table 1. Percentages of Quality Messages for WAP -based and WEB-based Forums 



WEB-based discussion was introduced in the midst of forum 3. The introduction of the WEB- 
based forum encouraged online participation and it did not negate all WAP-based postings. Five students 
continued to visit the WAP- and WEB-based forum at the same time. This suggested that the WAP-based 
forum complemented the WEB-based forum. The pre -service teachers were able to use WAP-enabled 
mobile phones or computers to participate in the WAP- or WEB-based forums. In this case, the advantages 
of both tools could optimize and enhance the online discussions. In other words, WAP technology 
provided another alternative to the online discussions, allowing opportunities for further collaboration and 
social interaction. During the interview, one of the pre -service teachers commented that she could better 
contribute to the discussion forums when she had a choice to use WAP-enabled phone if she was on the 
move or use her computer to log on to WEB-based forum when at home. The other pre -service teachers 
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said: “when you are on the move, at least you have the WAP-enabled phone to view the unread messages.” 
Generally, the discussions in both the WEB- and WAP-based forums were rather subject-focused, content- 
related and constructive. Irrelevant messages were minimal. 

The language that was used in the WAP-based forum discussion was a unique one. It was a 
feature found in neither formal writing nor does it resembled the messages found in the WEB -based forum. 
In WAP-based message, the number of short forms used was more frequent and perspective taking was not 
obvious. Although statistical results suggested that WAP-based forums produced more low-level quality 
messages than WEB-based forums, the potential and the contributions of WAP technology should not be 
ignored. In the survey that was conducted at the end of the course, 65% of the pre -service teachers agreed 
that WAP technology helped to build a closely knitted community. They believed that the WAP-based 
forum has formed a closely knitted group, and everyone was able to participate actively and learn from 
each other. By looking into the language pre -service teachers used in the WAP-based forum, we might be 
able to understand why WAP-based forum helped to build a closely knitted community. Some of the 
common short forms that were used in WAP-based forums were: “stu” for “students”, “n” for “and” 

“chem” for “chemistry” and “2” for “to” etc. For someone to be able to understand the syntax of these 
messages , one must be part of the community long enough to learn how to represent own ideas in such 
unique ways that only members of this community could understand. 

Another interesting phenomena found in this study was that pre -service teachers and class 
instructor, adopted different roles subtly. In many other situations, pre -service teachers in the class also 
played the role of a mentor by giving constructive suggestions and comments. One of the pre -service 
teachers created threads in order to better guide the rest when discussing. Although their class instructor 
did not assign such roles to the students, they have adopted diverse roles subtly to facilitate their online 
discussions. The majority of the pre -service teachers also agreed that the class instructor played a crucial 
role in the online discussions and that he had fulfilled his job as mentor, guidance and listener. 

Conclusion 

Learners’ changing characteristics prompt us to look into other new modes of course delivery. 

This study revealed the potential of WAP technology as an effective online communicating tool when 
coupled with other tool. It also documented the pioneering efforts of using WAP technology in the online 
discussions. More studies are needed to explore the possible ways of making WAP technology a successful 
social and intellectual tool for mediating individual learning and enhancing the social construction of 
knowledge. 
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No Kidding -Exploring the Effects of Stories through the Window of 

Schema Theory 
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Introduction 

We encounter stories everyday and everywhere. They are the oldest and most natural form of 
sense making (Jonassen, Strobel, & Gottdenker, 2004). It is a powerful tool that enables us to gain 
knowledge, understand phenomenon, remembers the unusual and to interact with the people around us. In 
this paper, we propose that stories can create an enormous impact on children, particularly those who are 
just entering the elementary school. 

Based on Schema theory, we intend to model the impact of stories have on young children, in this 
case, the 7 years old learners. The model created is one that demonstrates how stories can enhance the 
quality of schemas children possess. The quality of schemas will then in turn enhances the level of 
interpreting, predicting, and understanding stories. 

Stories, Schema 

Current research shows that reading stories to children is important (Allison & Watson, 1994). 
Some studies examining the influence of the teacher when doing reading to a whole class discovered that 
teacher's reading style affects children’s comprehension of stories (Dunning & Mason, 1984). Storytelling 
today is increasingly recognized as having important theoretical and practical implications and it is part of 
the emerging fields of discourse and narrative analysis (Kim, 1999). In education, storytelling is always 
recognized as an important factor as it reflects moral standards, life-styles, fantasy, humor, emotions, and 
different ways of knowing (Kim ,1999). Mandler (1984) distinguishes between stories, scripts and scenes 
and mentions that they have much in common and result in common types of psychological processing and 
they are represented in the human mind by related schematic forms of organization. According to her 
explanation, stories are literary expressions that we hear or read and are often refer to times long past or to 
imaginary worlds. Scripts represent the everyday events which fill our daily lives and scenes represent 
places which our daily routines take place. For the sake of the model we present in this paper, we shall 
refer all the three mentioned by Mandler (1984) as stories. 

Story telling helps us to understand human interaction, enhance our experiences and acquire the 
knowledge about the world we live in. We argue the importance of storytelling not only in school context 
but also at home. Family stories are important to the family as it involves the creation and maintenance of 
relationships, depicts rules of interaction, and reflect beliefs about family and other social institutions 
(Fiese, & Sameroff, 1999). The stories that are encountered among peers are also crucial as it plays a 
motivating context for literate behavior, as children communicate through narration to their peers in social 
play (Kim, 1999). According to Glenberg and Langston, 1992), comprehension of a story appears to result 
in multiple mental representations. One of these is a representation of the context, that is, a representation 
of its words and sentences. Another may be a mental model of what the context is about. And the 
representational elements of the mental model stand for such things as ideas, objects, events, and processes. 

Humans receive incoming information and organize it around their previously developed 
schemata, or what is called “networks of connected ideas.” (Slavin 1988). 

Schema theory was developed by, R.C Anderson, an educational psychologist. This theory views 
organized knowledge as an elaborate network of abstract mental structures which represents one's 
understanding of the world. We use schemas to interpret and also predict situations occurring in our 
environment (Widmayer 2003). Each individual has their own unique sets of schemas and these are built 
based upon individual's experiences and cognitive processes. Based on schema theory, learners acquire 
knowledge through three different reactions: accretation, tuning, and restructuring. In Accretation, when 
learners encounter information, they assimilate it into their existing schema without making changes to 
their overall schema. In tuning, as learners come across a situation whereby their existing schema is 
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inadequate for the new knowledge, they will then make modifications to their existing schema accordingly. 
Lastly, learners restructure and create a new schema when there is inconsistency between the old schema 
and the newly acquired knowledge. 

Our model seeks to explore how stories can create an impact on the quality of schemas children 
possess through the window of schema theory. 

Why Modeling? 

Models have been at the forefront of research in the philosophy of science for the last two decades 
(Guala, 2001). According to Lesh and Doerr (2003), models are conceptual systems that are expressed 
using external notation systems, and that are used to construct, describe, or explain the behaviors of other 
systems -perhaps so that the other systems can be manipulated or predicted intelligently. They explicitly 
explained that such a system consists of elements, relations, operations, and rules that govern the 
interactions within the system. 

Models are extensively used in many different fields, such as aeronautical engineering, 
agricultural sciences, business management or cognitive psychology (Lesh, & Doerr, 2003). The model 
we present in this paper is what it is called a dynamic model. A dynamic system is different from static one 
as such that the latter can only represent a system at rest, but the former can also represent the time- 
evolution of the system (Guala, 2001). Moreover, a dynamic system is also capable of performing 
simulation, that is presenting the intricate relations between different entities. 

The model (see figure 1) we presented using the software “STELLA” is one that depicts the 
intricate relations of variables and concepts. Our model displays sets of graphical representations that will 
interpret our understanding on how the quality of schemas is affected by stories encountered. 




Figure 1. Model that shows the impact of stories on children’s quality of schemas 
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Let the Model Runs! 

The subjects identified are young children of age 7. This particular group of learners is selected for 
we assume that they have just finished their pre-school education and are entering the elementary school 
with the curiosity to explore new things and acquire new knowledge. At the same time, they have some 
prior social interaction knowledge in their pre school education. 

The strength of this model is that it allows users to test hypothesis. Users are able to construct their 
own hypothesis and manipulate the variables and to run the models. This provides users the flexibility to 
make predictions and allows the users to make reasonable assumption on understanding the complexity of 
the system. Figure 2 shows the types of slider bar we have embedded in the model to allow users to 
manipulate the variables. 



forming rate 




Figure 2. Slider bar that allows users to manipulate the variables 

Before running this model, users have to make three assumptions. They will input the amount of 
the concepts in each story, the growth rate of same concepts, and the growth rate of different concepts. 

In this model, the children received stories from three main sources: Family members, teachers 
and friends. We started with assuming that more stories are told by family members before they enter 
school. Later when they begin school, we make assumptions that since they are just entering school, the 
probability to hear stories from their teacher compared to their family will still be lower and young children 
who have little skills in socializing, therefore the average numbers of stories they hear from their peers will 
be lower than from their teachers. Figure 3 depicts this relation. 




Figure 3. Children’s sources of stories 



After receiving stories from the three sources mentioned earlier on, children will start to process 
and decode them. They will relate the familiar stories with the concepts they have already possessed and as 
such, accretating (no change to the existing schemas) takes place. The assumption here is that young 
children who have just entered school are rather inquisitive and curious about the environment and 
learning. Hence, they will be eager to explore and acquire new information and knowledge. In the event of 
learning new information, the probability of forming new concepts will be higher. When new information 
enters their minds, it will then go through the process of either tuning (modification of existing schemas) or 
restructuring (forming new schemas). We supposed more new knowledge will go through the process of 
restructuring as kids are curious in nature. The chances of forming new schemas are higher than the 
chances of modifying their schemas. 

Next, when a child is able to interpret and predict the story, his level of understanding the story 
will increase. The other two factors that will also contribute to the child's level of understanding are the 
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quality of schemas and also the amount of prior knowledge the child possess. As we hypothesize that the 
quality of schemas will have a greater impact on the level of understanding, we give the increasing rate 
from the quality of schema a higher value than the values for the increasing rates from accretating, 
prediction and interpretation. As one increases the level of understanding, he/she will be able to relate the 
concepts learned from the stories to his existing concepts. 

As mentioned before, the quality of schemas is enhanced through three processes, accretating, 
tuning, and restructuring and the latter with the highest input value as it tends to form greater impact on the 
quality of schemas. Finally, the quality of schemas will determine the children's ability to interpret the 
stories. This means that the more quality schemas they possess, their ability level to interpret the stories 
will increase and hence their ability to predict the stories will also increase. Since prediction can only take 
place when one has already interpreted the story, the impact of interpretation must be higher than 
prediction. 

We do not intend to further discuss how cognitive process impact problem solving ability as this is 
not the main purpose of our model. However, the model did show that children’s ability to engage in 
cognitive processes remains constant at first but increases exponentially at a later stage. At almost the 
same time, problem solving ability also increases, and we want to highlight the fact that quality schemas do 
impact problem solving ability (see Figure 4). 




Figure 4. The impact of the level of quality schemas on the level of problem solving ability and cognitive 

process 



The Impact of Stories on the Quality of Schemas 

Our analysis shows that stories can improve the quality of children's schemas, and that their level 
of interpreting, predicting and understating of stories will be enhanced. Therefore, we propose that the 
more stories young children encounter, the higher the opportunity of enhancing their schemas. This will in 
turn enable them to better interpret, predict and understand stories. In one of our graphs (see figure 5), it is 
apparent that over a period of 12 years, the level of quality schemas and the level of understanding of our 
intended subjects improved at almost the same rate. 
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Figure 5. The impact of of quality schemas on the level of interpretation, prediction, and 
understanding 

Although we do not attempt to focus on how schema theory may apply to problem solving ability, 
we would like to state that one's problem solving ability is determined by the quality of schemas and the 
cognitive process. In figure 4, we see that the level of problem solving ability of our subjects will gradually 
increase. This is especially significant at the 10th year. We believe that as young learners gain more 
experiences in life and becomes more mature, their problem solving ability will therefore increases at an 
increasing rate. 

Our analysis also shows that the quality level of schemas and cognitive process will determine 
one's ability to solve problems. When children are exposed to stories, they do not only increase their 
schemas but also enhance their existing schemas. These enormous experiences which they encountered 
over time enhance their problem solving ability cumulatively. According to the study done by Price and 
Driscoll (1997), more than half of the learners are able to solve similar problems in a familiar context. This 
means that with more exposure to experiences (in this case, stories), children will be more capable of 
solving problems. 

Our model helps to make a significant prediction, that is, when children encounter more stories 
during their growing up, their schemas of the world will increase. 

Although computer modeling may not provide a perfect answer to our queries, it does provide an 
avenue for us to describe phenomenon, to make prediction on complex systems and to help us better 
understand them. We strongly argue that system modeling is a way to systemically study what we don’t 
know and reassure what we already know. 
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Designing a Peer Rating System for Asynchronous Discussion 
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Abstract 

The development of technologies enables us to create online learning environments. As online 
learning is expanding its role in higher education, it becomes one of fundamental issues how to create 
online learning environments to facilitate online interaction. One of the methods is using a discussion 
board where learners can interact with other participants. A peer rating system can help (a) motivate 
learners to actively participate in discussion, (b) provide learners with opportunities of assessing other 
discussions believing that assessment is a tool for learning, and (c) assess learners’ participation in terms 
of quality, not quantity, reducing the faculty’s workload. This paper will explore a way to facilitate and 
motivate learners to actively participate in asynchronous discussion by using a peer rating system. 

Although spoon-feeding education used to be considered a good way to instill students with facts 
and information, it is now regarded as impossible for teachers to feed knowledge to students in the era of 
information. Rather, searching for the right information, managing information, and making it meaningful 
knowledge should be the fundamental objectives of current and future education. Accordingly, learning 
should be considered as a process for constructing knowledge, which is to make information meaningful 
and relevant, not as a process of the simple transmission of information. 

According to Mayer (1992), there are three views of learning and instruction: learning as response 
acquisition, learning as knowledge acquisition, and learning as knowledge construction. From the 
perspective of constructivism, the third view, learning as knowledge construction, means highlighting “the 
interaction of persons and situations in the acquisition and refinement of skills and knowledge” (Schunk, 
2004, p. 287). Constructivism regards learning as learners’ active construction of knowledge. A lot of 
constructivists have been focusing on theories and strategies for helping learners construct their own 
knowledge representations. 

Along with that, instructional designers have been studying how to create learning environments 
in which technology supports the construction of knowledge. More specifically, since it is obvious that 
online learning environments differ from traditional classroom environments, existing classroom teaching 
methods are unlikely to be effective in online learning environments. Accordingly, new teaching methods 
and techniques for online learning environments should be developed to help learners interact with contents 
and others and to construct knowledge for themselves. The purpose of this paper is to find a way to help 
learners construct knowledge in online learning environments, motivating them to be actively engaged in 
online dis cussion through a peer rating system. 

Online Learning Environments and Interaction 

Just as Brown and Duguid (2002) say that “New technologies increasingly complement classroom 
and campus-based practices (p.XXII),” the development of technologies enables us to create Web-based 
learning environments. However, it does not mean that we simply transform traditional classrooms to the 
Web. Rather, “new and better ways are derived from constructivist theories and their related concepts such 
as knowledge-building, meaning-making, collaboration, and authentic, relevant, and student-centered 
learning” (Murphy, 2003, p. 1). 

Online learning environments can be used as a supplement to face-to-face instruction, in a blended 
mode with face-to-face instruction, and in web-based instruction without face-to-face instruction (Mishra, 
2002). Currently, a number of universities and colleges offer courses and programs through course 
management systems (CMS) such as Blachboard and WebCT. In 1998, 1.5 million students were estimated 
to have taken part in online courses (Smith & Winking-Diaz, 2004) and the number of students taking 
online courses is rapidly increasing. 

Online learning environments can provide learners with such advantages as flexibility in time and 
place, and easy access to learning materials. Bonk (2002) conducted a survey of online training to 201 
trainers, instructional designers, training managers, and human resource personnel in 2001. He identified 
that 86 % of the respondents were primarily interested in Web-based learning because it increased access to 
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learning. Also, ability to track learner progress, the standardization of content and assessment procedures, 
greater flexibility in delivery, and learner satisfaction were considered as advantages of online learning. 

One of the most important advantages is that online course can “create greater opportunities for 
instructors and students to interact more frequently, communicate more effectively, and collaborate on 
learning projects and research” (Smith & Winking-Diaz, 2004, p.l). Even though critics usually argue that 
interactivity is the missing element in online learning, it is broadly believed that current technologies can 
support effective interaction through learning experiences (Muirhead, 2001). 

However, “interactivity does not simply happen because the materials and tasks are presented to 
students for their consumption” (Smith & Winking-Diaz, 2004, p.3). There are a number of challenges in 
online learning environments to both teachers and students. First of all, teachers tend to overlook the 
amount of time required to create and maintain courses and evaluate students’ performance, but they 
shortly realize that they do need significant amounts of time to provide effective interaction in online 
learning environments. In addition, statistics shows that student attrition from online learning is relatively 
high (Hodges, 2004; Smith & Winking-Diaz, 2004). Usually, in online learning environments, students are 
likely to feel isolated resulted from the lack of immediate social interaction. Furthermore, learners need to 
change themselves to more active and self -directed learners, not passive ones (Muirhead, 2001). 

Asynchronous Discussion 

According to Milheim, “on-line interactivity has the potential of enhancing the quality of distance 
education, while improving student interactivity to create a climate that supports cooperative learning, 
critical thinking activities, and meaningful tutor/student academic collaboration” (Muirhead, 2001, 
Importance of Interactivity Study section, ‘|[. 1). Then, how can we implement the idea of interaction in 
online learning environments? 

A common method to increase interaction in online learning is asynchronous discussion where 
learners can exchange ideas and interact with other participants. There are a number of reasons that online 
discussion is a critical part of distance learning. The processes and activities of exposing to opposing or 
multiple viewpoints, sharing and negotiating interpretations, group reflection, and collaboration with others 
result from “many-to-many patterns of interaction such as what might be promoted through opportunities 
for dialogue, conversation or discussion” (Murphy, 2003, p. 2). Wiley (2002) mentions the following: 

From pragmatic matters such as the degree to which social interaction lowers student drop out 
rates, to pedagogical considerations around the depth of understanding students gain by 
negotiating the collaborative solution of problems to simple increases in student satisfaction with 
online courses due to opportunities for socialization, encouraging dialogue among our students 
increases learning in a variety of domains and meta-domains, (p. 1) 

However, it is much harder to facilitate online discussion and measure learners’ contribution to the 
discussion than it appears. Often, “groups of learners on online courses, in common with other online 
communities, are generally found to comprise both highly participative individuals and those who appear to 
contribute little to group discussions” (Williams, 2004, j[ 1). In addition, Dennen (2002) states the 
following: 

All aspects of a course discussion — initiation, facilitation, conclusion, and feedback — require 
different approaches when an asynchronous medium is used. ...There is no traditional instruction 
analog to asynchronous discussion, and thus this new medium needs to be examined closely in 
order to generate knowledge that will help online instructors learn and make informed decisions 
about how to design and facilitate asynchronous course interactions, (p. 1) 

A Peer Rating System 

Several studies shows that online discussion often fails to adequately engage students in learning 
(Smith & Winking-Diaz, 2004). Even though student interaction is a key for fostering learning by 
integrating personal experiences into class discussions and gaining insights from other students, it is often 
found that many students are not actively engaged in online discussion, providing and receiving online 
feedback (Muirhead, 2001). Furthermore, as the number of students taking an online course increases, 
teachers do not have enough time to read and evaluate all the postings. Then, how can we design 
asynchronous discussion in which learners actively participate? And how can we assess their participation 
without increasing the faculty’s workload? 
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For these issues, a peer rating system on online discussion can be applied, where learners post 
their ideas, questions, and reflections, read others’ postings, and rate them according to provided 
assessment rubrics. Peer rating is one form of peer assessment. Peer nomination and peer ranking are other 
forms of peer assessment. The peer assessment has been observed as a method to make students learn more 
and think more (Falchikov, 1986; Boud, 1989; Fry, 1990; Williams, 1992). Also, peer rating can improve 
students ’ writing skills through writing their own works and reading others’ works (Pope, 2001). 

The peer rating system in online discussion is a way to facilitate asynchronous discussion which 
provides learners with opportunities of collaboration, sharing, exploring multiple viewpoints and 
perspectives. This process can help learners reconstruct their own knowledge at the same time that they 
provide a valuable assessment service. Some activities that learners are expected to do for the online 
discussion with the peer rating system include; 

1. Preparation; Learners are expected to read materials or listen to lectures for discussion. 

2. Posting; Learners are expected to actively post their ideas, questions, and reflections based on 

readings and lectures as well as respond to the messages of others. 

3. Reading others’ postings; Learners are expected to read others’ postings. 

4. Rating; Learners are expected to rate others’ postings according to provided assessment rubrics. 

5. Management of their own learning; Learners are expected to enhance their postings to attain 
higher rating scores. 

According to McLoughlin and Luca (2000), “There is a new wave of pedagogy advocating 
‘alternative assessment’ in which assessment is integrated with learning and learning processes with real- 
life performance as opposed to display of inert knowledge’’ (Alternative assessment using technology 
section, 

f. 1). They argue that the authentic assessment is based on constructivism and the learners are 
regarded as the chief architect of knowledge building. They designed a course unit called “Interactive 
Multimedia Development Methodologies’’ and created peer-supported learning tasks which consisted of 
four processes; Criteria used by students for peer assessment, Student reflections on problem solving 
strategies, Student opinions of the assessment methods used, and finally Student perceptions of peer 
feedback as support for learning. From the study, they found that the integration of technology into an 
innovative assessment approach resulted in deep learning and students were engaged in active and 
reflective learning. Finally, they concluded that “the technology helped to foster the processes of learning 
by exposing learners to multiple views, achieved by assessment design and online discussion’’ (Discussion 
section, f. 3). 

The peer rating system is expected to help (a) motivate learners to actively participate in online 
discussion, (b) provide learners with opportunities of assessing other discussions believing that assessment 
if a tool for learning, and (c) assess learners ’ participation in terms of quality, not quantity, without 
increasing the faculty’s workload. 

Motivation in Online Learning 

Motivation is a key element for successful learning and teaching. According to Schunk (2004), 
motivation can be determined as “the process of instigating and sustaining goal-directed behavior’’ (p. 329). 
That is, highly motivated students are more likely to be engaged in activities that facilitate learning. 
However, motivation students in online learning environments can be challenges to instructors because of 
technology, isolation, poor communication skills, English as a second language, and lack of connection 
between content and students' needs (Beffa-Negrini, Cohen, & Miller, 2002). 

Many researchers have been searching for strategies or techniques to motivate learners in learning. 
However, it is believed that motivational strategies and methods in online settings are different from those 
in face-to-face classrooms. According to Bonk (2002), relevant and meaningful materials, timely and 
responsive feedback, goal-driven and product-oriented activities, personal growth, interactive and 
collaborative activities, engaging in discussion that involves multiple participants and a supportive 
community of learners are regarded as some of motivational principles. 

The peer rating approach can help motivate learners to actively participate in asynchronous 
discussion. When learners have more control to their learning, intrinsic motivation is likely to be increased 
(White & Weight, 2000). Also, the rating scores for final grades can play a role to increase extrinsic 
motivation. Since being expected to control and manage their rating scores within online discussion and 
compare their own postings with others, learners are likely to be motivated to increase their scores by 
interacting with others with better postings. 
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Online Peer Feedback 

According to McLoughlin and Luca (2000), many socio -cultural theorists highlight “the 
importance of reciprocal understanding and transactional dialogue where knowledge is exchanged and 
modified in the light of peer feedback” (Alternative assessment using technology section, 2). Peer rating 
can help learners in online learning environments have “feelings of group cohesion, loyalty and belonging” 
(Pope, 2001, p.242). 

The peer rating system can provide learners with opportunities of interaction among other 
participants. As stated by Mayer (1999), “Many of the popular instructional methods for promoting 
constructivist learning depend on interpersonal learning environments that enable discussion, modeling, 
guided discovery, and scaffolding” (p.152). Even though “the absence of face-to-face, peer, or teacher 
interaction possibly leads to negative educational experiences because of social isolation and working in an 
apparently impersonal environment” (Muirhead, 2001, Importance of Interactivity Study section, ( |[. 2), the 
peer rating system can improve interactivity between students and teachers and among students by 
providing opportunities of giving and receiving online feedback. 

Assessment on Learners’ Participation 

The peer rating (assessment) approach regards a learner as an active person. Students are expected 
to take responsibility for their learning through online discussion. As stated by Sluijsmans, Brand-Gruwel, 
Merrienboer, and Bastiaens (2003), “Assessment approaches promote integration of assessment and 
instruction, seeing the students as an active person who shares responsibility, reflects, collaborates and 
conducts a continuous dialogue with the teacher. . . .The emphasis shifts to a representation of assessment as 
a tool for learning” (p. 23). 

Furthermore, the peer rating system can reduce the faculty workload of grading learners’ 
participation. Rockwell, Schauer, Fritz, and Marx (1999) identify obstacles that discourage faculty from 
developing online education. From the study, they found “faculty tend to see distance education as a time 
demanding activity” (Obstacles To Teaching Via Distance section, f. 1). According to Dennen (2003), 
“some of the major complaints of online instructors in higher education are that the workload is high, 
particularly with regard to communicating with students, providing grades and feedback, and making sure 
students feel like they are connected” (p. 1). However, the peer rating system can automatically grade 
learners’ participation in online discussion in terms of scores. The scores can be directly used for a part of 
the final grades or as an incentive. 

This approach cares about the quality of postings, not how many postings learners contribute. 
Currently, many instructors grade online discussion based on each participant's overall number of postings 
and responses. To measure quality of learners’ participation, significant time and effort would be required 
because human raters have to review the contents of individual messages (Wiley, 2002). However, in the 
peer rating system, all participants can be human raters for reviews of other messages. In addition, they will 
also gain credit by participating in rating comments. 

Conclusion 

We live in the era of information. It will become increasingly important to search for right 
information, to manage information, and to make it meaningful knowledge. Because of the development of 
technology, we can take online courses without commuting to campus. Thus, the number of students taking 
online courses is rapidly increasing. 

In online learning, interaction has been a pivotal component for success. Accordingly, 
instructional designers should discover strategies and techniques by which learners are able to be actively 
engaged in courses. Relevant and meaningful materials, timely and responsive feedback, and interactive 
and collaborative activities are good examples. 

The peer rating approach introduced in this paper is expected to motivate learners to actively 
participate in asynchronous discussion by giving them opportunities to interact with other participants, 
providing others feedback and rating them according to provided rubrics. In addition, this approach also 
can reduce the faculty’s workload of grading learners’ participation. 
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Introduction 

Today, teacher education programs are faced with a variety of challenges. They must prepare 
future teachers to meet national and state standards with regard to both content and pedagogy in an era 
when there is increased emphasis on performance. Further, they must also help pre-service teachers learn 
to use technology and develop their understanding of diversity and multiculturalism to function within the 
changing schools of today (ISTE, 2000; NCATE, 2001). In this new climate, teacher preparation 
institutions must consider new ways of doing business, and technology offers capabilities that may help to 
address these challenges . 

Computer-based technologies have already transformed many aspects of work and daily life, and 
these changes are impacting education as well. As a result, computers and the Internet have become the 
focus of major educational initiatives and reform efforts, such as the U.S. Department of Education’s PT3 
program. Preparing Tomorrow's Teachers to use Technology, which aims to impact teaching and learning 
by improving the preparation of teachers to use technology effectively in the classroom. The focus on 
technology in the PT3 program stems from its potential to positively impact education, not just as a “bolt- 
on” to what already exists, but as a vehicle for making transformative changes in teacher preparation 
(Ertmer, 2003). 

In part, the PT3 program was created in response to the fact that the use of technology in colleges 
of education has been sorely lacking in the past. Several national reports over the past decade have 
bemoaned the poor state of teacher preparation with respect to technology use (Moursand & Bielfeldt, 

1999; Office of Technology Assessment, 1995; Panel on Educational Technology, 1997; Smerdon et al., 
2000). Problems cited include limited use of technology in teacher education courses, an emphasis on 
teaching about technology rather than teaching with technology, lack of faculty modeling, insufficient 
funding and faculty professional development opportunities, and lack of emphasis on technology in 
students' field experiences. 

To address these problems , colleges of education have begun to change their practices to embrace 
effective use of technology. Recommended practices include; (a) institutional planning for integration of 
educational technology into teaching and learning, (b) technology integration across the teacher preparation 
curriculum rather than limited to stand-alone courses, (c) increased opportunities for student teachers to use 
technology during field experiences, and (d) faculty development to bring about appropriate modeling of 
technology uses in their courses (Moursand & Bielfeldt, 1999). After more than five years of reform 
planning by its faculty and administration, the School of Education at Purdue University recently 
completed implementation of completely restructured elementary and secondary teacher education 
programs that make significant strides toward addressing these recommendations. A PT3 implementation 
grant, P3T3: Purdue Program for Preparing Tomorrow's Teachers to use Technology, in effect from June 
2000 through May 2004, provided substantial support for the implementation of these reforms. The grant 
project, its major initiatives, and some of its outcomes are described in this paper. 

Background 

Purdue University's new teacher education programs were launched with students entering teacher 
preparation programs in the fall of 1999, and the final new courses were put into place in spring of 2002. 
The new elementary and secondary education programs feature a cohesive set of courses, arrayed in a 
series of blocks, with practical experiences accompanying each block. The programs are anchored by four 
thematic strands - technology, field experience, diversity, and portfolio assessment. 

The technology strand is anchored by a required, introductory level, educational technology course 
that focuses on helping students build basic technology knowledge and skills within the context of 
planning, implementing, and evaluating instruction (Newby, Stepich, Lehman, & Russell, 2000). In 
addition, instruction in the application of technology in specific disciplines and with a variety of learners is 
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integrated throughout block and methods courses. Technology also provides a supporting infrastructure for 
communication, engagement, and reflection on practice. The field experiences strand is supported by 
Theory Into Practice (TIP) components that accompany each block of courses in the new program. The 
TIPs provide more cohesive field experiences for our students than were available in the past. The 
diversity strand is supported by appropriate course work and by exposing pre -service teachers to various 
forms of diversity (e.g., socioeconomic, rural/urban, religious, cultural, intellectual, special needs/gifted 
populations) during field experiences. The portfolio strand is supported by a new requirement that all 
teacher education students develop a professional portfolio in electronic format. 

Purdue's P3T3 project addressed each of these s trands and so played a significant role in 
implementation of the new teacher preparation programs. The overall goals of the P3T3 project were to: 

(a) prepare pre -service teachers to demonstrate fundamental technology competencies, using technology as 
a tool for teaching/learning, personal productivity, communication, and reflection on their teaching; and (b) 
prepare teacher education faculty in Education, as well as selected colleagues in Science and Liberal Arts, 
to teach pre -service teachers in technology-rich environments, modeling approaches that future teachers 
should use themselves when they teach K-12 students. The project sought to meet its goals via three 
complementary components: (a) a faculty development and mentoring program designed to assist the 
faculty in learning new technologies and effectively modeling their use in teacher education courses; (b) the 
development of a dynamic electronic portfolio system that provided pre -service teachers with the means to 
digitally represent their teaching performances; and (c) technology-enabled (virtual) distance field 
experiences for pre -service teachers in diverse setting. Ultimately, it was our hope that pre -service teachers 
would learn about technology, see it modeled by their instructors, reflect on their own learning about 
teaching using digital technologies, and, in the end, use these technologies for teaching and learning with 
their K-12 students. Together, these components addressed many of the challenges that confront Purdue 
and many other colleges of education. 



Faculty Development 

The faculty development component of the P3T3 project focused on helping faculty to acquire and 
refine technology knowledge and skills that they could use and model for the prospective teachers in their 
classes. The professional development component of the P3T3 project involved a two-day "start-up" 
workshop, technology skills development workshops, mini-grants, and a year-long support/mentoring 
program for participating faculty members. Approximately 95% of the faculty in the School of Education, 
along with selected teaching assistants and colleagues in the Schools of Science and Liberal Arts, 
participated in the project over its four years. 

The two-day start-up workshops, offered during summers and others breaks in the academic 
calendar, initiated participating faculty members and others into the project. Designed for about 10-20 
participants each, a total of nine start-up workshops were conducted for a total of 1 13 participants. In part, 
the start-up workshops were designed to model problem-based or inquiry learning processes as described 
by Torp and Sage (1998). Technology was used as a tool in this inquiry process and, additionally, was the 
subject of the investigation itself. For our workshops, participants working in small groups addressed the 
question, "What technologies are available at Purdue University to support teaching and learning, how can 
they be used, and what do faculty and students need to know about them?" Teams developed their own 
specific investigations, gathered information, and prepared multimedia reports about their investigations to 
present to the other groups. Technology was used during this process to acquire background information 
(e.g., Internet), produce artifacts (e.g., digital camera photos), and prepare a presentation (e.g., Powerpoint). 
Through this process, faculty members were exposed to inquiry -oriented approaches to technology 
integration. They were able to participate in the process, reflect on the roles of teachers and learners, and 
see applications of specific technologies in the classroomthat they might employ in their own classes. 

Following the inquiry activity, we demonstrated a variety of available technologies to raise 
awareness. Faculty members need to see models of what is possible in order to stimulate ideas for how 
they might integrate technology into their own classrooms (Ertmer, 1999). We examined examples of 
technology integration in K-12 classrooms, and we asked the faculty to reflect on potential uses of 
technology in their own teaching. Finally, we asked each participant to develop and share concrete plans 
for integrating technology into at least one course that he or she would teach during the coming academic 
year. This engendered commitment and gave the faculty member a clear goal to focus his or her efforts. 
This planning activity was the culmination of the start-up workshop. 
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Following each start-up workshop, and at various other times through the academic year, we 
offered hands-on, skills development workshops for participating faculty members. Topics included: 
WebCT (the “standard” web-based course support system at Purdue), web page development (e.g., 
FrontPage, Dreamweaver), working with graphics, concept mapping with Inspiration software, digital 
video capture and editing, IP-based video conferencing, and others. These workshops were designed to 
help the faculty develop the technology knowledge and skills they might need to better integrate the use of 
technology in their own teaching. In total, there were almost 900 enrollments in these workshops, and 
participants' overall evaluation ratings of the workshops were: Great - 66%, Good - 25%, OK - 3%, Fair - 
0%, Poor - 0%, No Rating - 6%. 

During the project's second year, we introduced Techie Talk, a less formal and briefer faculty 
development session. Techie Talks were 30-60 minute presentations or mini-workshops conducted during 
a weekday lunch hour during the academic year, faculty and staff could just drop in. We typically offered 
6-12 Techie Talks each semester. Some Techie Talk sessions focused on specific technology skills (e.g., 
tips for using email or MS Word), while others showcased faculty success stories related to technology 
integration (e.g., WebCT for course support, using IP-based video conferencing to connect with K-12 
schools). They offered a means of making connections among faculty and between faculty and the P3T3 
staff in a format that was more abbreviated than a full workshop. 

At the end of the second year, we also launched a mini-grant initiative. Each faculty member who 
participated in a start-up workshop received several hundred dollars of supply and expense funding to 
support their technology integration efforts. While helpful, we found that some faculty members were not 
able to do as much as they wanted with these funds and others had little use for the support. So, we took 
unused support dollars and created a pool of funds for a mini-grant competition. Faculty members 
submitted proposals for technology integration initiatives that were competitively awarded during two 
rounds of mini-grant funding. This led to a number of exciting faculty -developed initiatives. For example, 
Professor Brenda Capobianco, a science teacher educator, obtained two mini-grants to support the 
integration of technology in her elementary science methods course. With the mini-grant funds, she was 
able to purchase a set of electronic laboratory probes and accompanying software, which she introduced to 
support inquiry -oriented laboratory activities in her course. She transformed a course that had been largely 
devoid of technology into one in which technology was infused in support of a key course theme of 
inquiry -oriented science teaching and learning (Capobianco & Lehman, 2004). 

Finally, to assist the faculty in carrying out their plans and developing their own expertise, we 
offered a year-long support and mentoring program. Brand (1998) noted that despite increased access to 
computers and related technology, educators often experience difficulty in integrating technology into 
classroom teaching practice. Training and mentoring provide two major incentives in aiding faculty to 
successfully integrate technology in teaching (Dusick, 1998, Groves & Zemel, 2000). The P3T3 staff 
reviewed participants' personal plans for technology integration, and, based on the specifics of each plan, a 
graduate assistant with appropriate skills was matched to an individual faculty member to serve as a liaison 
with the project. The graduate assistant contacted the faculty member and offered support throughout the 
year, either working directly with the faculty member or, when necessary, referring the faculty member to 
another person with appropriate expertise. Support was provided through one-on-one tutoring and 
assistance at the faculty member's request. In addition, the P3T3 staff offered a drop-in help session one 
afternoon each week throughout the academic year for faculty members who were working on technology 
integration projects and needed immediate assistance. We viewed this support as critically important to the 
successful implementation of the faculty’s technology integration initiatives. 

What was the impact of the faculty development initiative? Based on the results of faculty and 
student surveys conducted at the beginning of the project in 2000 and again at the end of the project on 
2004, there was a substantial increase in the use of and comfort with technology. Faculty reported 
increased use of spreadsheets, presentation software, video conferencing, and hand-held technology. 
Students reported increased use of word processors, spreadsheets, web browsers, email, presentation 
software, digital cameras, and hand-held technologies. Whereas only 43% of students agreed that faculty 
used technology in classes at the beginning of the project in 2000, 99% of students agreed that the faculty 
used technology in classes by the end of the project in 2004. According to faculty self-reports, all of the 
responding faculty members reported that they integrated technology into their teaching, and 85% reported 
having changed their curriculum to add or increase the integration of technology. These data suggest that 
faculty members successfully integrated technology into teacher education classes. The most widely 
reported uses of technology were for: email communication with students (98% of responding faculty 
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members), Internet information retrieval by students (85%), in-class presentations (73%), online course 
resources (69%), technology-based assessment and evaluation (63%), and WebCT course support (54%). 

Clearly, many of the technologies used by the faculty involved the Internet and World Wide Web. 
Given that the Internet has become pervasive in K- 12 schools (Kleiner & Lewis, 2003), university teacher 
educators must model its use to help prospective teachers see effective ways to integrate it in their own 
classrooms. Many P3T3 faculty development initiatives focused on Internet technologies (e.g., web page 
design, WebCT, IP -based video conferencing). For the faculty members in our project, the attraction of the 
Internet, in part, was its ability to make information instantly accessible to others. As one of the 
participating faculty members commented n an interview, “I use the web a lot in my work area... this 
university is a highly technology involved university. Both of these are motivations for me to create a 
website for myself.” Another faculty member also cited the communication capabilities of WebCT by 
noting: "[It allows us] to extend the instruction beyond the classroom. We’ve been able to put up articles 
that students can look at and read. . . outside the regular class. That’s beenreally helpful, really useful." 

Faculty members saw the Internet as a tool for better connecting with their students. They used 
the Internet to communicate with students via email and through posting of course information online. 

They often created in-class assignments focused on information retrieval from the Web, an activity that 
mirrored their own professional use of the Internet as tool for keeping up-to-date on research and their 
discipline. They also used online course discussions in WebCT as a way of extending classroomdialogue. 
These uses of technology create opportunities for what Dede (1996) has called distributed learning in which 
the technology facilitates communication and collaboration. While the technology was not viewed by most 
faculty members as a replacement for conventional approaches, the faculty in the P3T3 project at Purdue 
embraced those uses of the technology that complemented their classroom interactions by facilitating the 
building of connections with their students. 

Dynamic Electronic Portfolio System 

Portfolios are another tool for the building of connections in teacher education. Portfolios are 
purposeful collections of student work that demonstrate effort, progress, and/or achievement. According to 
Danielson and Abrutyn (1997), portfolio developers engage in four processes: (a) collection - the gathering 
of relevant materials, (b) selection - identification of those materials that best demonstrate knowledge and 
capabilities, (c) reflection - thinking about one's own practices, and (d) projection - looking forward to 
consider what steps need to be taken to improve. Through this process, teacher candidates grow and 
develop, and the resulting portfolio provides a richer picture of their understanding than can be achieved 
through more traditional, objective measures. Portfolios provide a vehicle for pre -service teachers to 
demonstrate their understanding of teaching and learning and so connect their own learning to the standards 
that guide teacher certification. 

There is growing interest in the use of electronic multimedia portfolios for documenting growth 
and development of pre -service teachers (Barrett, 2001; Read & Cafolla, 1999). Electronic portfolios, or e- 
portfolios, have advantages over their paper counterparts including the ability to represent materials in 
multiple ways, ability to link to standards, reduced storage demands, accessibility, and students' 
development of technology skills in the process of creating the portfolio. E-portfolios can be created using 
tools ranging from off-the-shelf generic computer applications to a customized application built specifically 
for that purpose (Barrett, 2001). In the P3T3 project, we focused on the latter by creating our own 
customized, large-scale, electronic portfolio system. The system was designed to allow our pre -service 
teachers to collect and archive relevant example of their work, submit selected work for faculty assessment, 
and receive feedback from the faculty about it. The system was designed to support a direct connection 
between ongoing assessment and reflective practice. 

The Purdue Electronic Portfolio (PEP) system was housed on a server with about two terabytes of 
storage space, enough to give each one of our approximately 2000 pre -service teachers the storage 
equivalent of a CD-ROM. Candidates' artifacts were stored in a Microsoft SQL Server database, a popular 
choice for large-scale, web-accessible databases. Candidates interacted with the system through a web- 
based interface driven by Microsoft Active Server Pages (ASP) technology. Because it was web-based, 
candidates could access the e-portfolio system from any place with an Internet connection. 

Pre -service teachers could log in to the PEP system, manage their account information, upload 
files, and create artifacts. They could upload most digital files — word processing documents, photos, 
scanned images, Powerpoint presentations, even videos. Any individual item of evidence was stored in a 
file. In our parlance, an artifact was an individual file or collection of files that the student assembled in the 
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PEP system to address one or more professional standards. Thus, an artifact could be a single thing (e.g., a 
written lesson plan) or a set of related things (e.g., a written lesson plan, a grading rubric for use with it, a 
photo or video of the candidate conducting the lesson in a K-12 classroom). Students used a template to 
create an artifact; the completed artifact was a secure web page with links to associated files. Each artifact 
included common elements — the student's name and photo, course information, relevant standards — as 
well as whatever components the student wished to include. Students could classify artifacts according to 
three broad themes developed by the Purdue faculty (attention to learners, understanding curriculum in 
context, and commitment to professional growth) and according to the ten 1NTASC principles that 
undergird many teacher preparation standards. Students could add and format their own components to 
personalize artifacts; these components could be accessed by the student or instructor through live links on 
the resulting web page. Artifacts, finally, could be assembled to make portfolios. See Figure 1. 



Files Artifacts Portfolio 




Figure 1. Organization of the Purdue Electronic Portfolio System 



After creating an artifact, the pre -service teacher released it to an instructor for evaluation. Until 
released, an individual artifact remained private and could only be accessed by the pre -service teacher who 
created it. The system also allowed students to make their artifacts public, allowing other individuals 
within the PEP system to view it. (No artifact was completely public, because access to the PEP system 
required a login ID and password.) 

Faculty members could log into the system to assess students' work. The instructor could retrieve all of the 
students' artifacts for a particular course during a particular semester for assessment. In order to track 
students' progress and growth as they proceeded through the teacher education program, another layer of 
assessment corresponds to review of the overall portfolio. In Purdue's assessment system, this overall 
review occurred at four points in the student's academic career. At each of these checkpoints, or gates, 
students must demonstrate appropriate progress on the portfolio to proceed in the teacher education 
program. 

In general, students have found the PEP system to be relatively easy to use. However, conceptual 
barriers have been more substantial. Teacher education students at Purdue had not been required to 
produce portfolios previously, except in individual courses. As a result, the idea of creating a longitudinal 
portfolio throughout their programs of study was unfamiliar. Further, most students were not comfortable 
with the idea of assessing their own proficiencies with respect to established state and national standards. 
Faculty guidance in addressing these issues will be critically important as we move forward. 

Of the developmental issues that have been encountered, among the most significant has been 
determining how students should pass through the multiple assessment gates that correspond to the key 
assessment points of Purdue’s reformed teacher education programs, and who should monitor this process. 
The gate review mechanisms, assessment rubrics, and procedures were determined by the faculty. But, a 
tension existed between the des ire of the faculty on the one hand to ensure that students created an 
integrated and reflective portfolio that cut across courses and the desire on the other hand to minimize the 
extra effort involved in assessing the work of hundreds of teacher candidates . After much discussion, the 
faculty agreed upon gate review procedures that placed the responsibility for particular gate reviews within 
the context of key courses within the program. While this may limit some of the connections between 
courses and concepts that we seek in our teacher preparation programs, is was manageable . 
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While students, for the most part, readily adapted to the PEP system, the faculty was less 
comfortable with it. Nonetheless, PEP acted as a catalyst for change. The demands of imp lementing this 
new system have forced the faculty to address the creation of appropriate artifact-producing assignments 
across the curriculum, procedures for gate review, and rubrics to be applied to all teacher education 
students. The ensuing discussions have been good, and, as a result, Purdue's programs have improved. In 
addition, many faculty members have sought to improve their own technology skills to keep pace with 
students who are creating multimedia electronic portfolios. 

Students have come to recognize that the portfolio is something of great personal and professional 
importance that will follow them throughout their college careers and probably beyond. However, they still 
have difficulty with the notion that they are responsible for understanding what is expected of them, as 
defined in state and national standards, and how they must show in their own work that they meet these 
expectations. The idea of creating these connections does not come naturally to most students; it is 
something we as teacher educators must help them to do. This is a challenge, but it is one that we gladly 
accept, because it is through these connections that we build a stronger teacher education program. 

Technology- Enabled Field Experiences 

Field experiences have been identified as a key means to better prepare teachers for the diversity 
and complexity of today's classrooms (Goodlad, 1990). While field experiences are generally recognized 
as critically important, many colleges of education, particularly those in rural areas such as Purdue, have 
difficulty placing students in field settings that provide for needed experiences with, for example, diverse 
student populations. Distance education technologies offer capabilities that can provide needed 
experiences for pre -service teachers when appropriate field sites are not in close proximity. Using distance 
education technologies, specifically video conferencing and the Internet, future teachers can observe and 
also interact with K-12 classrooms from afar. This concept is certainly not new. As long ago as the 1960s, 
closed circuit television was used to enable teacher education students to observe school classrooms (e.g., 
Abel, 1960). In the 1980s, Iowa State University’s Teachers on Television program established that the 
observation skills of preservice elementary teachers could be improved through remote observation of 
public school classrooms using microwave-based video connections (Hoy & Merkley, 1989). However, 
closed circuit and microwave television technologies were expensive and difficult to set up and maintain. 
Today’s video conferencing technologies offer a much more flexible and cost-effective option for 
observation of and interaction with school-aged learners at remote school sites. 

To date, most literature about video conferencing has dealt with traditional distance education in 
which course content originating at one location is delivered to students at other locations. More recent 
literature has discussed other educational applications of video conferencing. For example, some schools 
have experimented with virtual field trips, typically short term experiences where K- 12 or college students 
connect to a distant site by means of video conferencing to learn more about the site or participate in a 
planned activity (LeBlanc, 2002; Pachnowski, 2002). In Indiana, for example, a number of K- 12 school 
sites are connected to an intrastate fiber optic video network called Vision Athena 

( http://www.visionathena.org ), managed by the Center for Interactive Learning and Collaboration, a partner 
in the P3T3 project. Using the Vision Athena network, K-12 classes are able to connect to the Indianapolis 
Zoo or the Indianapolis Children’s Museum to learn about exhibits and interact with educational personnel 
at these sites. As part of our P3T3 project, we explored another application of video conferencing as a tool 
to link teacher education classes with diverse K-12 students and classrooms for observation and 
interaction. Although a few such projects have been reported (Edens, 2001; Howland & Wedman, 2003; 
Phillion, Johnson, & Lehman, 2004), this is an application of the technology that remains relatively little 
explored. 

The P3T3 project implemented an initiative to use video conferencing technology to support 
distance or virtual field experience for pre -service teachers and develop various models for enhancing 
teacher preparation through linkages between the university and participating K-12 schools. Four Indiana 
school districts were partners in the project: School City of East Chicago, Crawfordsville Community 
Schools, Lafayette School Corporation, and Lawrence Township Schools of Indianapolis. While all four 
partner districts offered some types of diversity, two in particular - East Chicago, an urban community in 
northwest Indiana, and Lawrence Township, in the Indianapolis metropolitan area - had student 
populations that were more ethnically and socio -economically diverse than those in most of the schools 
near the Purdue campus. 
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At the outset of the P3T3 project, we expected to use the Vision Athena network for the video 
conferencing. While we did use that network on a limited basis, IP-based video conferencing equipment 
from Polycom ( http://www.polvcom.com) emerged during the project as a better way to meet most of our 
needs. This technology supports good quality video and audio over the Internet, is relatively affordable, 
and is very flexible because a standard H.323 Internet video conferencing connection can be established 
between any two points with reasonably fast (128 Kbps or better) Internet access . Special distance 
education rooms or video studios are not needed. Room-to-room video conferencing was supported by 
Viewstation units, which start at about $2,500. The Viewstation has an integrated camera with panning and 
zooming capability that can be attached to any available video monitor and plugged into an Ethernet jack 
for Internet connectivity. With a hand-held controller, users can control the remote end camera to focus in 
on particular students or activities in the distant classroom. For person-to-person or small-group-to-small- 
group connectivity, we used Polycom ViaVideo desktop video conferencing units, which operate in 
conjunction with a Windows PC. While its camera is of lesser quality and lacks the panning and zooming 
capability of the larger Viewstation units, the ViaVideo is inexpensive (about $400) and supports 
application sharing during video conferencing. Using ViaVideos, pre -service teachers could tutor 
individuals or small groups ofK-12 students. 

The most extensive pilot project involving the use of video conferencing was conducted with 
beginning teacher education candidates in Block I, in which teacher candidates take two classes: (a) 
Exploring Teaching as a Career and (b) Multiculturalism and Education. The two foundational courses 
share a theory into practice (TIP) early field experience, in which the pre -service teachers ordinarily travel 
to nearby schools to observe classrooms for a couple of hours each week to observe teachers, teaching, 
schools, and student diversity. Because Purdue is not located near a major urban center, opportunities for 
the pre -service teachers to encounter diversity are limited. In addition, the pre -service teachers sometimes 
feel there is little need to understand diverse populations of students because they expect to teach in 
predominantly white and rural areas after graduation (Yao, 1999). However, the demographics of 
communities and the schools that serve them are rapidly changing and diversifying (Glazer, 1997). This 
pilot project was designed to help our teacher candidates experience the ethnic, linguistic, and socio- 
economic diversity they need to be prepared for the future. 

In this pilot project, beginning pre -service teachers enrolled in one or two sections of the TIP did 
virtual rather than actual field experiences through the use of video conferencing and the Internet. Each 
semester for eight semesters, Professor JoAnn Phillion and her students linked with a teacher and students 
in an elementary school in a diverse inner city school in East Chicago using Polycom equipment. The class 
of teacher education students connected with the bilingual elementary classroom about once a week 
throughout the semester for between one and two hours each session. During that time pre -service teachers 
observed the classroom, interacted with the children and teacher, and prepared and presented a variety of 
enrichment activities. 

Prior to beginning the video conferencing field experiences, the university class visited the 
participating school at which time the pre -service teachers toured the school; met staff, teachers, and 
students; and interacted with the students in the class involved in the project. This visit allowed the pre- 
service teachers to gain first-hand knowledge of the school and the students, which we believe helped to 
overcome the impersonal nature of video conferencing communication. After the site visit, the virtual field 
experiences began and continued weekly through the semester. Initially, pre -service teachers spent time 
observing the classroom and getting oriented to classroom activities. Unlike the typical TIP experiences in 
the class, in which individual students visited different classrooms during the week and discussed their 
observations during the next class, the video conferencing allowed all of the students in the class, as well as 
the instructor, to observe of the same classroom setting. This created a shared context for discussions about 
the classroom and the actions of the teacher. As the semester progressed, the pre -service teachers became 
increasingly involved in actually interacting with the students in the distant classroom. 

A typical interactive session began with the classroom teacher teaching a lesson. Pre -service 
teachers then took turns, individually or in small groups, teaching enrichment or reinforcement mini- 
lessons to the students. These enrichment/reinforcement activities were worked out in advance in 
consultation with the classroom teacher to supplement her existing curriculum. Over the life of this pilot 
project, pre -service teachers taught lessons on fractions, story books, historical figures, and the 9/11 World 
Trade Center disaster. In one semester, Purdue pre -service teachers were invited to prepare activities 
related to Japan, a year-long theme in the elementary classroom The pre -service teachers, in groups of 
three, prepared lessons related to Japan’s geography, school life, food, daily activities, wildlife and 
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art/drama/literature. A variety of enrichment lessons were created, adding to the teacher's curriculum while 
giving the pre -service teachers a chance to learn about and work with diverse elementary students. A video 
program describing this pilot project was produced by Soundprint Media and WHRO-TV, the PBS affiliate 
in Norfolk, Virginia, for the Teaching Now! (formerly PT3 Now!) video series and can be viewed online at: 
http://teachingnow.org/watchTV.php?id=30 . 

In addition to the pilot project described above, a number of other experiments in the use of the video 
conferencing technology were conducted. For example. Professor Mark Balschweid used video 
conferencing to allow students in his agriculture teacher education class to unobtrusively observe a 
classroom to reflect on what goes on there. As with Professor Phillion’s pilot project, the video 
conferencing permitted an entire class of pre -service teachers to observe a K-12 classroom setting 
together, thus creating a shared context for discussion about what they saw. Professor Tristan Johnson, 
a faculty member in educational technology, used video conferencing and the Internet as a vehicle for 
his instructional design class to create and deliver instructional materials to an audience of K- 12 
students. In one semester, the university students developed a website and video-based lessons about 
cartoonist Rube Goldberg, metric measurement, and simple machine concepts to capitalize on a well- 
known Rube Goldberg machine contest started by engineering students at Purdue. Video conferencing 
sessions were used to introduce students to concepts that built toward the culminating activity of the 
lesson, a Rube Goldberg machine building contest for the 5 th graders (O’Connor, 2003; Phillion, 
Johnson, & Lehman, 2004). Most recently. Professor Gerald Krockover, a science educator, used 
video conferencing to supplement face-to-face supervision of student teachers. In each case, video 
conferencing supported extensions or enrichments of the traditional teacher education curriculum by 
enabling university classes to reach out and connect with K-12 classrooms at a distance. 

As part of the project evaluation, students in Professor Phillion’s pilot project were surveyed about 
their perceptions. Table 1 shows the results from end-of-semester Likert -type survey items from one year. 
While students had little prior experience with the technology, 69% agreed or strongly agreed that they 
were comfortable with the video conferencing equipment by the end of the semester, and 81% agreed or 
strongly agreed that it was easy to use. A large majority (81%) agreed or strongly agreed that they learned 
to use the video conferencing equipment from this class experience. About two -thirds (66%) agreed or 
strongly agreed that the technology was a valuable addition to the class. A majority (60%) agreed or 
strongly agreed that that they felt more comfortable using technology for teaching and learning as a result 
of the class experience, and, significantly, 83% agreed or strongly agreed that that they felt more 
comfortable teaching diverse learners as a result of the experience. 



Table 1 Pre-service teachers’ responses to end-of-semester video conferencing survey items (n=42). 



Survey item 


Strongly 

Agree 


Agree 


Undecided 


Disagree 


Strongly 

Disagree 


By the end of the class, I felt comfortable with 


6 


23 


12 


1 


0 


the video conferencing equipment that we 
used. 


14% 


55% 


29% 


2% 


0% 


The video conferencing in this class was easy 


8 


26 


8 


0 


0 


to use. 


19% 


62% 


19% 


0% 


0% 


I learned how to use video conferencing in 


6 


28 


3 


4 


1 


education from this class. 


14% 


67% 


7% 


10% 


2% 


I believe that the use of video conferencing 


11 


17 


6 


4 


4 


was a valuable addition to this class. 


26% 


40% 


14% 


10% 


10% 


Because of the experience in this class, I feel 


9 


20 


9 


2 


2 


more comfortable in my ability to use 
technology for teaching and learning. 


12% 


48% 


21% 


5% 


5% 


Because of the experience in this class, I feel 


13 


22 


4 


1 


2 


more comfortable in my ability to understand 
and teach diverse learners. 


31% 


52% 


10% 


2% 


5% 
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One benefit of the experience seemed to be the development of pre -service teachers' classroom 
observation skills. Consistent with the finding of Hoy and Merkley ( 1989), the beginning teacher education 
majors came into the course as unskilled observers, but through the guidance of a faculty member who 
observed alongside them via the video conferencing, they became better observers themselves. In addition, 
the shared observational context led to opportunities for richer class discussions. The university students 
generally felt that the remote field experience was instructionally valuable, increased their confidence, 
better prepared them for teaching in the future, and engendered a desire to continue using technology for 
teaching. They also showed evidence of significant growth in their understanding of diverse students and 
how to teach them. One pre -service teacher commented, “I also think that being able to see a more diverse 
classroom than the ones close by was a big advantage for us because it gave us something to relate our 
multicultural studies to.” Another remarked, ‘T've learned not to be afraid of teaching students in the more 
run down communities . . . they're not as scary as I had first imagined.” 

Of course, there were difficulties that must also be acknowledged. Most schools are protected by 
an Internet firewall which must be configured to allow selected outside connections. When trying to set up 
video conferencing connections, we ran into difficulties that required time and effort on the part of 
technical support staff to resolve. The IP-based video conferencing technology is good but sometimes 
technical problems or teacher absence caused the cancellation of a video conferencing session. In addition, 
the connections sometimes became “choppy” or broke up as a result of limited bandwidth or network 
congestion. Even when working perfectly, it was difficult for pre -service teachers to make observations by 
watching a video screen, and the children’s voices were difficult to hear over the background noise of the 
classroom Furthermore, the main issue for the pre -service teachers was that they were not in a “real” 
classroom with “real” students. Some students felt cheated that they did not get to go into an actual 
classroom each week. However, most of the future teachers seemed to benefit from the experience. 

When we consider all factors, these virtual field experiences seemed to be a worthwhile way to 
expose pre -service teachers to experiences they might not otherwise get. Our teacher education program 
has at its core emphases on early and continued field experiences, on developing technological skills, and 
on understanding diverse learners. Virtual field experiences offered a way to expand the options for linking 
teacher education students with K- 12 teachers and students. While we do not advocate replacing traditional 
field experiences with virtual field experiences, these experiences do seem to offer significant potential for 
augmenting the experiences of prospective teachers in university preparation programs. 

Conclusion 

Teacher education is faced with a variety of challenges today. Technology offers new capabilities 
that can enable teacher education institutions to do a better job of meeting those challenges. Technology 
makes possible the development of connections, and those connections make possible new ways of 
addressing traditional problems in teacher education. 

New technologies, particularly those associated with the Internet, can enhance communication 
between teacher educators and future teachers. Using the web, via course websites or course portals such 
as WebCT and Blackboard, teacher educators can provide a central point of information for teacher 
education students. Further, they can use electronic mail and online discussion system to extend office 
hours and in-class discussions. These approaches break down the traditional boundaries between in-class 
and out-of-class time and experiences. As a result, there are opportunities for the development of richer 
dialogue and the growth of true communities of learners in teacher education programs. 

Electronic portfolios offer opportunities for other kinds of connections. Future teachers today 
must demonstrate that they have acquired the knowledge, skills, and dispositions to be effective teaching 
professionals. Portfolios offer a means by which students can document in rich and varied ways their own 
growth, development, and ultimately competency. Electronic portfolios allow teacher education candidates 
to build a collection of their work using familiar, easily accessible, and easily editable digital tools. 

Further, their work can be connected to those state and national standards that must be satisfied to obtain 
teacher licensure. Thus, teacher candidates today can use technology not only to build a professional 
resume but also to show how that resume satisfies society's demands for what teachers must be able to 
know and do. 

Finally, technology can be used to connect teacher education institutions with the K-12 schools. 
While field experiences have long been a part of most teacher preparation programs, they are often limited 
by available placement opportunities in the vicinity of the college or university. Video conferencing 
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technology offers a flexible tool with which teacher education institutions can connect with K- 12 schools at 
a distance. Such connections, while not a replacement for traditional field experience, offer opportunities 
for new experiences and for ways to introduce future teachers to students and settings that they would be 
unable to encounter otherwise. 
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Project Management for Web- based Course Development 

Transferring face-to-face courses into Web-based courses is a trend in higher education. Whether 
this course transition is for distance education or for resident instruction, faculty members play a critical 
role in the process. Faculty members not only provide lesson content, but important insights into how 
content has been best presented in classes semester to semester. However, faculty involvement alone does 
not guarantee a quality online course. It is the combination of faculty working with an instructional 
designer and the instructional design team that molds the content and personal teaching experience into a 
rich learning environment for the online student. Further, this transition process must be guided by a solid 
project plan that outlines major milestones for the faculty and team members. For without a solid project 
management plan content may not arrive when needed and resources cannot be scheduled to assure that the 
course is completed in a timely manner. Delayed or unexpected lesson content will lead to project cost 
overruns and missed deadlines. 

The Penn State World Campus 

The Pennsylvania State University has been involved in distance education since 1892 and has 
produced courses that have been delivered via a variety of media. In 1997 Penn State launched the World 
Campus as the primary delivery unit for courses offered to students at a distance. This year also marked the 
beginning of development of online courses for our distance education students. While courses in the past 
had integrated listserves and gophers, 1997 marked the beginning of the design and development of wholly 
online courses delivered through WebCT. As of 2004 Penn State’s World Campus offers more than 200 
Web-based courses and enrolls students from around the world. 

Responsibility for transitioning face-to-face courses to online course rests with the Instructional 
Design & Development (ID&D) Team of the World Campus. This team consists of a director, assistant 
directors, project managers, instructional designers, instructional designer specialists, graphic artists, a 
multimedia team, production specialists, technical typists and a technology team (programmers) who have 
different responsibilities. 

Authors of the Web-based courses are Penn State fulltime faculty members. These faculty 
members work with the instructional designers to determine learning goals and objectives, generate ideas, 
write the lesson content and storyboard, provide test items and exercises, and sugges t multimedia 
selections. 

It is the task of the instructional designers to develop the courses within a given timeframe and 
within budgetary constraints. This role takes on great significance as more academic units look to 
technology as a means of offering hybrid courses and wholly online courses in residence and at a distance 
to assist with student demand and a need for greater flexibility in scheduling. It is imperative that these 
ventures be done within budget and on time in order to show a return on investment for both the institution 
and the academic units. 

Project Management Models 

Over the years distance education at Penn State has had a tried and true project management 
practice for the development of print based independent learning courses. This model allowed for an 8 to 
12 month development cycle where faculty authors first met with the designers to review their existing 
face-to-face course and examine existing print based course. At the end of the initial meetings the faculty 
left with a course design guide in hand and constructed in writing their course content. This process took 6 
to 8 months. Once the faculty member had finished the draft of the content they once again met with the 
instructional designer who then worked with the faculty to tailor the course, learning activities, and 
assessment strategies to the distance education student. Once the faculty and the designer had crafted the 
course the final product went to the academic unit for approval and was then sent to the technical typists for 
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final preparation in the templated study guide format. However, as these courses were independent rolling 
enrollment courses they did not open to students until they were completely done. Thus, if timelines were 
missed it had little impact on student expectations and costs were contained as faculty were paid a flat rate 
and designers did not begin work on the courses until all content had arrived. 

As Penn State moved to begin the transition of face-to-face courses to online course it was 
perceived that a similar project management cycle would work for the online courses. Thus the original 
project management model for the design and development of online courses mirrored that of the print 
based courses with one semester being allocated to the development of content by the faculty and the 
second semester being dedicated to the production of the course in the WebCT environment. 

However, what worked well for the design and development of the print based courses did not 
translate well to the development of online courses. These online courses tended to be cohort semester 
based courses and part of integrated curricula. Therefore, it was often the case that the announcement of 
these courses preceeded final development and timelines for delivery were locked down. Thus, missed 
deadlines lead to delayed launches or courses starting when they were not complete. This added a great 
deal of pressure to the faculty authors, designers, and instructors. 

What was often experienced in the early development of the online courses using the two -semester 
model was delayed content delivery. As the faculty authors for the online courses were fulltime faculty 
demand on their time was extensive and the idea of sitting down to write a full course was often 
overwhelming. Thus, it was not all that uncommon at the end of the first semester course content was not 
complete and both the faculty and the design staff had to cut corners and work long hours to get the course 
finished by the end of the second semester. 

Upon examining these process failures it was determined that a better project management model 
needed to be implemented. This lead to a series of benchmarking visits with both corporations involved in 
the development of online courses and with other institutions involved in distance education. The final 
result was a project management model dubbed the two -week cycle model. 

Two Week Cycle Model vs the Two Semester Model 

As stated above the two semester project management model provided faculty with one semester 
to write and develop content, and then the design staff were given a second semester to develop the course. 
However, in several instances, content arrived late, thus pushing out the projected completion dates. In the 
past 18 months a new two -week cycle model has been adopted that adjusts expectations for the faculty in 
terms of content due dates, and allows the content to be mocked up and tested in a cyclical process. 

The Two -Week Cycle model allows designers to develop and get each lesson or unit of a course 
ready for review in two weeks. During the first week of each two-week cycle, designers work closely with 
faculty in order to get lesson content on time. Then during the second week the design staff mocks up the 
lesson online and prepares it for review by the faculty. Also, within the second week, faculty start writing 
the next lesson or unit of content. By the end of the first week of the next two week period, another content 
for another lesson is ready for the designer to develop and integrate into the Web-based course. If a Web- 
based course has twelve lessons, ideally, twenty-four weeks (six months) later, the course should be ready 
for final review and editing prior to opening. One of the key benefits of the Two Week Cycle model is 
designers receive content every other week, which keeps things moving. Another benefit is constant 
communication with faculty. He or she goes over design questions each lesson with the designer as the 
lesson is being developed and gets a real feel for the instructional design process. Faculty and designers can 
anticipate areas to modify in the lessons as the course unfolds which results in a better course when 
development is completed Thus, the two -week development cycle allows faculty to get each of the lessons 
in on time, which is the desired goal of the designer? 

While the conception of the two-week model is around a two-week cycle, designers have adapted 
this to meet the schedules of certain faculty. Some have adopted a three or four week cycle with two or 
three lessons due at the end of each cycle. Regardless of the length of the cycle, which should be no longer 
than one month, the process helps keep the faculty authors and the design team focused on the development 
needs and the timeline. 

Critical to the two -week cycle model is the first five weeks of the project management model. 
During this five week period five key things must occur. First the faculty must deliver a completed draft of 
their course outline or syllabus. This initiates the first design team meeting where the team discusses, with 
the faculty member, all aspects of the course. During this meeting the course is dissected and graphic and 
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multimedia elements are reviewed, readings are identified, and copyrighted material is discussed. At the 
end of this first meeting the faculty and design team have a good conceptual idea of how the course will be 
developed and what resources are required to complete the task. 

Following this first meeting the designer works with the faculty to mock up one of the lessons. 
This process provides further insights into design requirements and resource needs. Also, during this 
process all copyrighted material is identified that will need clearance. Upon completion of the mock up of 
the lesson the design team meets once again in the 5 th week to finalize the design, budget, and timeline. A 
product of this meeting is the final design document for the course. 

How to Work with Faculty Involved in the Development of a new Web-based 

The above has outlined the conceptual aspects of the two -week model. However, what are the 
tools employed that facilitate the process. Designers need to work with faculty efficiently and effectively to 
guide faculty through writing online lesson content and providing the necessary materials. In order to do so, 
designers may use the following steps: 

• create project management Gantt chart; 

• create mini Web site for the project management; 

• provide detailed course outline form with a sample; 

• provide a lesson content template with examples; and 

• make a regular communication plan. 

1. Create Project Management Gantt Chart 

It takes time to establish a long six month timeline using a calendar. With software, such as Microsoft 
Project 2003, one can easily create a Gantt chart which contains timeline, project tasks, names of who 
need to complete a specific task, task starting date and ending date, and task time duration, etc. Gantt 
charts allow a convenient way to make a detailed project management plan, as well as remind all of 
the team members what tasks they should do and completion deadline for each task. 

2. Create Mini Web Site for the project manaeement ( See Appendix A ) 

When you print out Gantt chart and want to share with the rest of team on the project management 
plan, it is not convenient to show people since the printed Gantt chart is too long. But you can easily 
create a mini Web site (two to three pages when printed) for this project management based on the 
Gantt chart with team member tasks and deadline for each task in it. Compared with Gantt chart, the 
mini Web site is easier and more convenient for team members to check what they should do and be 
aware of their tasks and deadline for each task so that they can plan their time ahead. Also a mini Web 
site provides a blue print of the project for the whole team. Moreover, it is easy to update in order to 
track a project. 

3. Provide Detailed Course Outline Form with a Sample 

Once faculty have a project timeline in hand, and understands what to do overall, it’s time for him or 
her to review how lesson content has been written for other distance education courses. The first thing you 
may want the faculty to draft is a detailed course outline. This provides faculty with a clear idea of what the 
final course will include. It can also serve as the basis for the syllabus for the course. 

Below is a typical course outline and shows what might be included: 

• Course description 

In this section, faculty may answer the following questions. What will be covered in the 
course? Will this course be an independent learning course, or will there be other students pacing 
through the course at the same time (as they would in a face-to-face class)? Will you expect 
students to interact with fellow classmates? Will you expect students to stick to a prescribed 
pace of study or can they work through the course at their own pace? 

• Course goals/objectives 

List 4-5 broad statements of what faculty hopes students will know, or be able to do, or 
have experienced as a result of taking the course 
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• Course prerequisites 

Let students know if there are course prerequisites for this course. 

• Outline of overall course structure 

The following questions will be answered in this section. How many lessons will the 
course be broken down? How much time will students spend to complete the course? 
How much time will students have to work through a single lesson? How much time dose 
faculty expect students to devote to the course each week. 

• Required course materials 

List any textbooks, articles, workbooks, videos, software, or other special materials 
students will need to have in order to complete the course. For each item, provide as 
much identifying detail as possible (such as ISBN number for a textbook or ordering info 
for a brochure). 

• Course requirementList the graded assignments for the course (e.g. papers, projects, 
quizzes, exams, class participation grades, etc.) with directions description student can 
follow to complete assignments, as well as with the percentage of the course grade that 
each assignment will be worth. 

• Each lesson specific objectives 
Objectives for each lesson are listed here 

• Proposed schedule 

Lesson titles, scheduled timeframe, related readings, and assignments will be listed. 

• Grading scale 

Let students know the grading policy, such as how many points are required for an “A” 
grade, so on and so forth. 

It is good practice to provide faculty a sample course outline from a real course to help them to 
understand how to draft his or her own course outline. 

4. Provide a lesson content template with examples 

With the course blue print — detailed course outline in mind, faculty can start to work on lesson 
content. After many years working with faculty, we have found that it is easier and really helpful if we 
provide faculty a lesson content template, as a lesson content template with examples lets faculty know 
what he or she should write without taking too much time to figure how to get started. Below is a 
lesson content template might be included. 

• Introduction 

• Lesson objectives 

• Reading assignment 

• Reading tips/summary 

• Lesson content/commentary /class notes 

• Lesson activities 

• Lesson summary 

5. Make a Regular Communication Plan 

A regular communication plan will allow the designer to work closely with the faculty, for example, 
weekly phone calls can save time in terms of tracking the project or solving problems. 

Conclusion 

In summary, designers need to work with faculty closely to meet tight project deadlines. When the 
designers spend time creating samples, template and detailed guide lines for faculty, this will save time and 
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avoid the need to go back and forth between designers and faculty during the course development process. 
With the above method to manage a project, the two week cycle model, and the above documents to guide 
faculty to write lesson content, the Web-based course project will be effectively and efficiently designed, 
developed, and hopefully meet the project deadline on time and budget. We hope that these ideas are 
helpful for you when you work with faculty to transfer a face to face course into Web-based course. 

Appendix A 

Pro j ect Managemen t —Time l ine 

Last updated: 10/3/2004 



<Home> <Pre> <Next> 



Tasks 


Person 

Responsible 


Duration 


Starting 

Date 


Deadline 


Status 


Notes 


Held first Author Meeting 




1 day 


Thu 11/20/03 


Thu 

11/20/03 


Done 




First Draft Detailed Course Outline to 
ID 




1 1 days 


Fri 11/21/03 


Fri 12/5/03 


Done 




Provide feedback on Detailed Course 
Outline to Author 




1 day 


Mon 

12/8/03 


Mon 

12/8/03 


Done 




Initiate Intellectual Property 
Agreements 




3 days 


Mon 

12/8/03 


Wed 

12/10/03 


Done 




Meet with Development Team to 
Discuss Activities, Ideas 




1 day 


Thu 

12/11/03 


Thu 

12/11/03 


Done 




Send Final Detailed Course Outline 
and Unit 1 to ID 




9 days 


Tue 12/9/03 


Fri 

12/19/03 


Done 




Send Unit 2 Content to ID 




16 days 


Sat 12/20/03 


Fri 1/9/04 


Done 




Convert Unit 1 to online environment; 
send it and Detailed Course Outline to 
the Program Manager for Approval 




15 days 


Mon 

12/22/03 


Fri 1/9/04 


Done 




ID Goes Through Unit 1 




2 days 


Mon 

12/22/03 


Tue 

12/23/03 


Done 




ID Sends Revised Unit 1 With 
Suggestions Back to the Author 




1 days 


Tue 

12/23/03 


Tue 

12/23/03 


Done 




Author Sends ID Unit 1 With 
Changes/Revisions 




9 days 


Wed 

12/24/03 


Mon 

1/5/04 


Done 




ID Looks Over Unit 1 Changes and 
Sends Final Unit 1 to Tech Typist 




1 day 


Mon 1/5/04 


Mon 

1/5/04 


Done 




ID Sends Graphic Ideas and 
Multimedia Ideas 




1 day 


Mon 1/5/04 


Mon 

1/5/04 


Done 




ID Put Unit 1 Activities in Angel 




5 days 


Mon 1/5/04 


Fri 1/9/04 


Done 




Typist Sends Final Unit 1 to ID 




2 days 


Mon 1/5/04 


Tue 1/6/04 


Done 




Graphic Designer Sends Graphics to 
ID for Review 




5 days 


Mon 1/5/04 


Fri 1/9/04 


Done 




Media Specialist Gives Multimedia 
Products to ID 




5 days 


Mon 1/5/04 


Fri 1/9/04 


Done 




Meet with Development Team to 
Finalize Activities, Ideas 




1 day 


Tue 1/13/04 


Tue 

1/13/04 


Done 




Create Ms ting of all Anticipated 
Copyright Permissions and Video 




4 days 


Mon 

1/12/04 


Thu 

1/15/04 


Done 
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Licenses Needed for Course and Send 
to ID 














Initiate requests for all necessary 
copyright permissions and video 
licenses 




1 day 


Fri 1/16/04 


Fri 1/16/04 


Done 




Convert Unit 2 Content to online 
environment 




4 days 


Fri 1/9/04 


Fri 1/23/04 


Done 




Same Processes as Unit 3~Unit 14 














Draft Welcome Page Site and send 
URL to Author 




16 days 


Fri 3/5/04 


Fri 3/26/04 


Done 




Review Welcome Page Site and 
provide revisions by Author to ID 




6 days 


Fri 3/26/04 


Fri 4/2/04 


Done 




Notify Program to approve Welcome 
Site and Unit 2-4 




5 days 


Mon 4/5/04 


Fri 4/9/04 


Done 




Send approval form for Welcome Site 
to ID 




6 days 


Fri 4/9/04 


Fri 4/16/04 


Done 




Draft Activity Sheet and Send 
Welcome Letter to Production 
Specialist 




32 days 


Fri 4/16/04 


Mon 

5/31/04 


Done 




Schedule Implementation in mid-July 




31 days 


Wed 5/5/04 




Done 




Provide Administration Training to 
Instructor 




6 days 


Fri 7/23/04 


Fri 7/30/04 


Done 




Provide Angel Training to Instructor 




6 days 


Fri 8/6/04 


Fri 8/13/04 


Done 




Notify Program to Approve Final 
Course 




1 day 


Wed 8/4/04 


Wed 

8/4/04 


Done 




Send approval Form for Final Course 
to Dong 




6 days 


Wed 8/4/04 




Done 




Finalize all Parts of Online 
Environment 




1 1 days 


Wed 8/4/04 




Done 




Notify ITs that it's Time to Move 
Course over and Link to University 
Registration System 




1 day 


Wed 

8/18/04 


Wed 

8/18/04 


Done 




Enable Angel 




1 day 


Fri 8/20/04 


Fri 8/20/04 


Done 





<Home> <Pre> <\ext> 
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CHESt - An Educational Tool that Understands Students' Questions 

Serge Linckels 
Christoph Meinel 
University of Trier, Germany 

Introduction 

New technologies are used in many courses and for many occasions. Good teachers try to use the 
best tool and the best method to introduce or to treat difficult subjects by presenting the information in 
different ways: spoken words, written text, pictures, graphs, movies or by using interactive computer tools. 
It is a fact that students who discover the solution to a problem by themselves, for instance by searching 
information on the web or by using a multimedia computer tool, understand and memorize the learnt 
subject matter better than if they simply listen to or read the information. In this sense, computer-aided 
learning, or e-learning, seems to become a more and more important and useful part in education. 

In this paper we present our "intelligent'' e-learning tool that understands students' questions. In 
the first section, we introduce our view of the advantages and disadvantages of e-learning and our vision of 
a perfect educational tool. In section 2, we present our solution CHESt, the Computer History Expert 
System with its multimedia interface, the idea of splitting the knowledge in a large number of small clips 
and its semantic search engine. In section 3, we describe in detail how the knowledge base is semantically 
described with RDF, the Resource Description Framework , by building a reservoir of CHESt vocabulary 
and by generating the CHESt dictionary and the RDF serialization. More technical details about the 
semantic search engine are presented briefly in section 5. Ideas for future and related work are presented in 
section 6. 

Historical view 

The e-learning vogue started in the mid 80s as a promising application of new technologies and 
inventions like the Compact Disc (CD), the Personal Computer (PC) and better graphic adapters and 
displays (for example: CGA, Color Graphics Adapter). Soon, it was said that these novel computer tools 
would replace teachers, especially when several years later, the first online lessons were broadcast via the 
internet. It is true that from those early visions, several real advantages for everyday education were born. 
Here some examples: 

For the student: 

• The multimedia aspect and the attractive interfaces attract the student's attention. 

• Courses are broadcast live or on demand over the Internet. Thus, students can review a missed 
lesson or an important topic before a test. 

For the teacher: 

• The teacher has the possibility to promote autonomous learning. 

• Distant learning is possible without displacement (for teachers and even students). 

Pitfalls In e-learning 

However, many e-learning tools and solutions are the results of theoretical and scientific research 
rather than of practical, concrete and founded needs in education. Consider the following two examples: 

• A multimedia encyclopedia is a great tool for teachers to find information for preparing their courses. 
But students can only use few immediately. The information is often presented in too complicated a 
language and there is simply too much information on one topic. There are no filter techniques to 
adapt the content to the skills of the user (for example: less information for a kid, exhaustive 
information for a teacher). Often, search mechanisms are based on simple keyword searches that are 
not effective, for example: the user gets a large number of possible results. 

• A lot of universities offer courses online, often streamed from a server. We will not discuss any 
performance or financial constraints; each lesson that has to be transmitted normally takes over an 
hour. Well, let us suppose that the student has to take into account a dozen of such lessons for a test. 
Even if he is searching for precise information, it is difficult and very time consuming to scan through 
all the possible streams to find the appropriate part. 
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Based on our teaching experience, we perceive the e-learning technologies as a complement to 
classical education and useful for special occasions. We do not agree to the fact that computer programs 
should become an equivalent substitution for teachers or classical methods. Students should primarily use 
e-learning tools at home, for example to review a certain difficult topic, to receive advanced or simplified 
information about a certain subject or to help them doing their homework. Otherwise, the abuse of 
computers and e-learning tools in classrooms will provide students with the misleading view that 
everything is so simple that it can be learned by "playing". As already mentioned above, an occasion where 
an e-learning tool is pertinent and useful in school is when a new topic is introduced. Here, the teacher may 
decide to let the students discover this new topic by themselves in order to attract their interest. 

A computer tool cannot exp lain a difficult topic better than a teacher. It can only present the 
information in another form, maybe a clearer or more exhaustive one. But it can neither understand the 
student's real problem nor provide further and different explanations adapted to the student's sense of 
perception. 

Furthermore, the interaction between machines and humans is still surprisingly complicated. 
Students often have problems to express themselves. Formulating their problem in a computer 
understandable form seems not to simp lify the problem. Clicking on some icons on the screen is certainly 
very simple, but doesn't allow the user to express himself freely. 

In direct consequence to the above mentioned problems, the heavy use of computer tools in 
education is more time-consuming than classical methods. For example, because web-based search engines 
work with keywords, the resulting list of websites is often very long and most entries are not really 
pertinent. Therefore, the student spends most of the time surfing on the web searching for information but 
without clear and precise results at the end and without acquiring new knowledge about the initial subject. 
Furthermore, most students are slow in typing text or in manipulating correctly specific software. Thus, if 
they had to use conventional tools like books to search for information, they would have found the answer 
to their question faster than with a computer tool. 

Our vision 

Our vision is to create an e-learning tool that should understand students' questions. The 
interaction with that tool should be as human as possible, maybe even by means of spoken words. The user 
should be able to freely formulate his question. The system understands his question and returns a precise 
answer in multimedia form. Here are some key features of our novel e-learning solution: 

• The tool can be used as a complement to normal courses (in the classroom or at home). 

• It does not require special hardware and can be used on any computer. 

• No installing or configuration procedures are necessary. 

• The answers are in a multimedia form. 

• The answers are taken from a secure knowledge base. 

• The knowledge base can be extended easily. 

• The tool promotes independent learning. 

• The student creates his own course content by assembling different multimedia answers. 

• The interaction between the user and the system is very easy. 

• The search for information is not limited to a simple keyword search. 
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The prototype CHESt 




Figure 1: Screenshot of the prototype CHESt with a keyword search on "zuse”. The window shows a list of 
search results in the bottom right-hand corner. Selecting a topic from this list will play the clip, like the one 
shown in this example, where the teacher uses an interactive board. Added handwritten comments made by 
the teacher are integrated and applied in real time on the text (top right-hand window). 

On the basis of the overwhelming experiences of using Tele -TASK (see section 2.1.) in university 
teaching, we started to investigate whether it can be used in (High) schools as well. Our research project 
started in 2003 in close collaboration with teachers, who tested our prototype in practice. Results and ideas 
were firstly published in [21] and [22], We focused on one general context, namely computer history. The 
aim of the project CHESt ( Computer History Expert System) is to design an e-learning tool for computer 
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history that allows pupils to easily find information by means of asking questions (see figure 1). The 
prototype is based on the following features: 

• Within CHESt , the knowledge is presented in multimedia form. 

• The content of the knowledge base is split into a large number of small clips. 

• A semantic search mechanism is used for information retrieval. 

The multimedia interface 

Today, kids are spoiled with all the wonderful and attractive interfaces of operating systems, 
applications and games. New software without a graphical user interface in vogue is banned to failure. 
That's exactly why students prefer websites with colors, images, sound and animations, rather than for 
example books as learning syllabuses. In fact, isn't it clearer and easier to read something that is illustrated 
with images, pictures or drawings? Every person is different in his sense of perception. Some understand 
better if they hear the explanation by the means of verbal communication, some need to write it down, 
others must see it in the form of a text or a picture and others again have to touch it. A good teaching tool 
must present the same information in different forms in order to activate as many senses as possible. The 
psychological foundations were proven by the work of [10] and [11]; information that is presented at the 
same time in different forms improves the understanding of the information. 



Figure 2: Schema of the CHESt user interface. 

The interface of our tool is basically organized in three windows (see figure 2). The first window 
(video and audio) shows a teacher explaining something on the whiteboard. This is the student's common 
view in a classroom and should create a kind of "virtual classroom" atmosphere. Based on practical 
teaching experience we can confirm that students often take lessons where they use a new computer tool or 
do research on the web for example, as a kind of game, without relation to the "normal" lessons. The video 
sequence should keep them concentrated on what they do and draw their attention to what the teacher is 
explaining. 

The second window represents the usual blackboard. It is in fact a zoom on the whiteboard that 
the teacher uses in the video (first window). Although the blackboard is the most used medium in schools, 
it has a lot of disadvantages, for example: 

• It is impossible to represent pictures. 

• It is difficult and time -consuming for the teacher to create a complex drawing. 

• It is time -consuming for students to reproduce its content in their books. 

• The content is not available for later lessons and must be reproduced. 
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The virtual blackboard in our tool has the following features: 

• The teacher can use this area for an on-screen presentation (for eximple: PowerPoint). 

• He can add handwritten information to the smartboard, which is reproduced in this window both 
simultaneously and in exactly the same way. 

• He can also display the desktop of his connected laptop, for example in order to explain a certain 
application, to show a website or to demonstrate the settings of the computer. 

The third window can be used for any purpose. It can contain links to a photo gallery, hyperlinks to 
additional information on the web, book references or just a single picture of the subject the teacher is 
speaking about. 




Figure 3: Tele-TASK architecture. 

We used Tele-TASK [3] [9] [12] to record the lessons in order to create one well-structured 
multimedia stream (see figure 3). The result is a RealMedia file that can be played with any compatible 
software, for example the free RealOne Player [6]. 

The clip approach 

Essential in our concept is the length of the stored items in the knowledge base; the duration of the 
video sequences. The younger the user, the shorter the time during which he/she will concentrate on the 
information displayed on the screen. Furthermore, we mentioned already in the introduction that it is not 
easy to find the appropriate information inside a large piece of data, for example an online lesson that lasts 
90 minutes. Therefore, we divided all our multimedia data into small clips. The duration of each clip varies 
from several seconds to 3 or 4 minutes. Each clip documents one subject or a part of a subject. Together, all 
the clips of the knowledge base cover one large topic. In our prototype CHESt ( Computer History Expert 
System), we focused on one precise topic: computer history. We produced 300 clips about every important 
event in computer history. CHESt exists as standalone application (we managed to store the whole 
knowledge base with the application software on one single CD-ROM) and as online application that can 
be found at [7], The later uses a streaming server to transmit the clips to the user's browser. 
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Splitting a large topic like computer history into a lot of small pieces is much easier than we 
assumed at the beginning. We are now convinced that most courses taught in schools or at universities can 
be divided into smaller atomic units where each covers one precise subject. Teachers of different fields 
confirmed that this concept is not limited to computer-science and that it could be used in their field too. 

For instance, in language courses, a teacher could record one clip per grammatical rule. Another concrete 
test was made in the field of biology where a teacher used our tool to explain the basic function of the 
heart. Further details would be explained in additional clips. 

One more advantage of that clip -approach is the simplicity of administration. If the tool does not cover a 
certain topic, a new clip can be recorded and added to the knowledge base. The intervention of a computer- 
science expert is not necessary. 

Finding the right clip 

Having a large knowledge base with short multimedia clips is one thing; another thing is to find 
the right clip. The more clips you have, the better your knowledge base covers a certain topic, but the more 
difficult it is to find an appropriated clip. A first solution is of course to let the user browse through a table 
of content where all the clips are listed in categories, for example; hardware/storing devices or people/still 
living , etc. and load the chosen clip. This possibility is offered in the standalone version of our tool, not in 
the online version for the moment. The main disadvantage is that there is no additional information about 
the content of the clip except for a short designation. Furthermore, this operation is time-consuming and not 
very effective, because the user has to search and maybe test different clips before he finds the answer. An 
automated search would be better. At the moment, the prototype CHESt has only a keyword search. If the 
user enters "arpa", the system will list all clips about the ARPA and the ARPANET. The user then selects a 
clip from that list to be played. The main disadvantage is that the user must already give a part of the 
answer. For example, you want to know who invented the first computer. Then you should enter keywords 
like "Zuse” or "Aiken". You cannot ask: "Who invented the comp uter?” Another problem is that, depending 
on the keyword, you will get a long list of possible results. Finally, even if a clip is about a certain topic, it 
must not necessarily be found from the keyword the user has entered, for example the user enters "disk" but 
the matching keyword would be "floppy". 

The most efficient search mechanism is to allow the user to enter a complete question. The tool 
should "understand" that question and give a small list of pertinent clips as answer, or better even just one 
clip. Technical details about a semantic search engine are described in section 3. This solution is also 
pedagogically welcomed because in schools, students are forced to express themselves in complete 
sentences and not just with keywords. Most important is the fact that the interaction between the student 
and the tool takes place in a very human and simple way. An imaginable improvement would be a verbal 
communication where the user could speak his question into a microphone. 

Example of an all day application of CHESt 

With the features described bellow, we could imagine that the student who is working with CHESt 
has his own virtual teacher. These teacher's answers are short and presented in an interesting multimedia 
form. The student can communicate with him in a very simple and human way by typing his question, or in 
a later improved version by means of verbal communication. As every tool and method, CHESt will not 
replace every conventional lesson. We see it as a complement useful for certain occasions. It's up to the 
"real” teacher to decide, for which of his lessons it is appropriated, for example; 

• To introduce a new subject by letting the students discover themselves some new information. 

• To use CHESt as complement to find illustrations for a certain topic (for examples pictures of old 
computers or computer pioneers). 

The students could work in groups or alone. In fact, they create their own course content: the clips 
they consult. Depending on the kind of work, they can print a certain scene of a clip, copy snapshots into a 
text document or simply take notes. The teacher is sure that the information they get is correct and secure. 
Here a concrete scenario: "Hi students. Today we are working on computer history. Here is a list of 
interesting questions. You have 40 minutes to search for information before we discuss your answers 
together. Of course, use CHESt!" . 

• Who invented the computer? When? 

• What is the Colossus? 

• What is a transistor useful for? 
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Explain the word FTP. 

Who sent the first e-Mail? 

What was the size of the first hard disk? 
Who invented Unix? 



Describing the meaning of the clips 

In section 2, we described our prototype CHESt from a pedagogical view. The search of a certain 
clip, not by keywords, but by a freely formulated question is one of the main necessary improvements. 
Though, before the tool can even try to understand the user's question, it has to "know" what data are stored 
in the knowledge base. In other words, every clip must be described in a machine -readable form. Therefore, 
we have to add data to each clip to describe its meaning. That kind of data is called metadata. For this 
purpose we use the Resource Description Framework (RDF), introduced by the W3C in 1998 to build the 
Semantic Web [16]. In principle, this is done once, at the moment when the clip is added to the knowledge 
base. However, the computer can assume a part of this task. The different steps are described below. 

The CHESt RDF vocabulary 

With our concept to use short clips, we have the great advantage that we can describe the meaning 
of one clip with few metadata. We divided the CHESt knowledge base logically into two classes: clips that 
describe inventions (things) and clips that describe inventors (persons). Assertion: an invention was 
invented by one or more inventors. An invention and an inventor can be a resource (in our case: a clip) or a 
value (just a textual information). Every resource is described with properties. An inventor has three 
properties (predicates): his name (vCard : FN), the year of his birth (chest : year_birth) and the year 
of his death ( chest : year_death); if still alive, this property is left blank. As you see, we used the 
W3C recommendation vCard namespace property full name (FN) [15]. The class invention is divided into a 
number of subclasses to better organize the different resources (see figure 4). We used the Dublin Core (dc) 
namespace [4] to describe an invention with the following properties (predicates): its description 
(dc : title), its date of first appearance ( dc : date) and its creator ( dc : creator). The complete 
CHESt RDF schema can be found at [8]. With these few elements we can semantically describe every clip. 




Figure 4: Class hierarchy of the CHESt RDF classes 

Generating the CHESt dictionary 

The next step is to search inside every clip for metadata. For example, the clip, which describes 
the calculator "ENIAC" should be scanned to find its description, the year where it was first taken into 
service, and the name(s) of its creator! s). We tried to apply an approved approach in the field of computer 
linguistics: create a dictionary of synonyms for every CHESt RDF element [5] [14]; in one column one will 
find the RDF elements and in the other column there is a list of natural language synonyms. For example, if 
we are scanning for dc : creator , we are searching for words like creator, builder, constructor, inventor, 
etc. For our prototype, we decided to consider only the textual data from the PowerPoint presentations and 
to ignore the teacher's audio information and his handwritten notes for example. With a special tool [17] we 
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are able to convert the PowerPoint documents into pure text files. Then the stemming process can begin. 

All non-words (words that contain digits or special characters) and words with just one letter were 
eliminated from the generated text files because they have no semantic influence. All words are converted 
into lowercase and special characters are replaced by a space. Finally a list of 20640 remaining words was 
created from the whole 300 clips in the knowledge base. All were represented in a tree, where every node 
represents one letter. The tree is built in less than a second. The words are read vertically from the top 
(root) down along the branches. This technique also allows to eliminate all double words. Each node 
contains the number of words that end with that particular letter. There are 4215 remaining unique words 
with an average length of 8.049 letters per word. 

The dictionary of synonyms is built from that tree. The idea is to regroup words with similar 
spelling and thus with the same meaning (for example: build, built, builds). It is impossible to detect 
automatically all synonyms, because there are words that have a similar spelling, but not the same meaning 
(for example: consult, consume). The aim of the stemming process is to limit human intervention by 
proposing clusters of generated synonyms. Further details of this process are described in [22]. 

Why didn't we use an existing dictionary of synonyms, for example WordNet [20]? For two 
reasons: first, by choosing an existing dictionary, CHESt would immediately be set to a certain language 
(English, German, French...). Our solution is language independent, because it builds its dictionary from 
an existing content. Second, even if we still have 4215 unique words to scan for synonyms and RDF 
elements, it is still much less than a complete dictionary with at least 200 times more words. Note also, that 
the words listed in our dictionary are words that are used at least one time. 

Generating the RDF description 

The final step consists in scanning through the clips (as text files) and searching for synonyms for 
the RDF elements described in section 3.1. In our case 273 out of the 300 clips were described 
automatically and without human interaction. In some clips, different concurrent synonyms were found. 
The most frequent example is the RDF synonym for dc : date which represents the date of first public 
appearance of an invention. For different inventions, there was a date of planning, a date of starting the 
construction and a date of launch. To solve this ambiguous problem, we programmed our tool so that, in 
case of concurrence, it chooses the second occurrence and protocols the problem in a log file. The final 
result is an RDF/XML serialization for each clip (see figure 5). We used Jena [18] to generate the RDF 
serialization. Jena allows to store de triples in a simple XML-file but it also supports several RDMS (for 
example MySQL or PostgreSQL). 



<?xml version="l . 0"?> 

<rdf : RDF 

xmlns :rdf="http : //nww. w3 . org/1999/02/22 -rdf-syntax-ns# " 
xmlns : dc= "http : / /www . w3 . org/2001/vcard-rdf/3 . 0# " 
xmlns : chest= "http: / /www . linckels . lu/chest/elements/1 . 0/"> 
<chest : Person 

rdf : about = " http : //sigma957 . lte. lu: 80 80/ramgen/Ar chive/ Zuse . rm"> 
<vCa rd : FN>Kon rad Zuse </vCa rd : FN> 

<chest :year_birth>1910</chest : year_birth> 

<chest :year_death>1995</chest : year_death> 

</ chest :Person> 

</rdf:RDF> 



Figure 5: Example of a semantic description of a clip using RDF/XML and streaming access to the 
multimedia files. The clip is about the person "Konrad Zuse". 

Understanding the user 

The number of results (in CHESt a matching result is a clip) will be shorter and more pertinent 
with a semantic search than with a normal keyword search. Furthermore, the user must not enter a part of 
the answer in its question, for example: “Who invented the first computer’' doesn’t contain the name of the 
inventor. In fact, the name of the inventor is the information to find. 
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What did Aiken invent? 




?x; <dc: creator>; "Aiken " 



Figure 6: Principle of the inference engine that transforms a non-formula question into a well -formulated 

RDF query. 

We now dispose of a well-formulated and semantically described knowledge base in RDF (see 
section 3). To perform a semantic search, the question entered by the user must be transformed into RDF 
too, in order to have the same structure for the question and for the database (see figure 6). The backbone 
of our semantic search is an inference engine which transforms a normal sentence (the user’s question) into 
a well -formulated RDF query. We used RDQL to access our RDF knowledge base [23]. See [1] [2] [19] for 
more details about semantic databases. For example: “What did Aiken invent?” should become: 

select ?x WHERE (?x; <dc : creator>; "Aiken") . 

As described in section 3.2, all the words in the dictionary are basically regrouped in two 
categories: words that are of any semantic use (which are associated with an RDF element) and words 
without semantic use (which are not associated with an RDF element). It is clear that this dictionary can 
only be used in a precise context, which is computer history in our case. The user's question is also put in 
that same context for parsing, for example if the users asks "Who invented the penicillin?" the tool cannot 
give an answer because the question is outside the tool's context. Starting with these constraints, the 
transformation of a common formulated sentence into RDF can be resumed by saying that the system has to 
replace all semantically important words by the RDF corresponding elements and to throw unimportant 
words away. Of course, the shorter the questions, the better the results. 

Since all RDF elements in the CHESt schema are defined either as {subject, object] or as 
{predicate} (see section 3.1), there is no doubt about the membership of the recognized RDF elements. 
Except chest : Person and chest : Invention (or one of its subclasses), all RDF elements are 
predicates. As we are dealing with questions, there should always be a missing part, normally the subject or 
the object. Remember the basic assertion: "An invention was invented by an inventor". Generally, members 
of the class chest : Person are objects, members of the class chest : Invention are subjects. 

Conclusion and Outlook 

Our primary aim is to create a tool or even a new method of teaching. The teacher is in the 
background and the student plays the role of an explorer. Therefore, it motivates the student because (s)he 
can create his/her own course content. The information is presented in an interesting multimedia form. The 
system "understands" the questions of the user and gives efficient answers: there are no long searches for 
answers, but the requested answers are rendered in a concise form. Of course, a motivated student is a good 
student and good students normally achieve better re suits. Thus, this tool is supposed to improve education. 

The prototype CHESt covers the field of computer history, but by generalizing the knowledge 
base, it can be used in nearly every course in any school, college or university. Its advantages are that it 
promotes independent learning. By adding other clips from other fields (for example: biology, electronics, 
etc.), CHESt could become more than just an expert system on computer history. Ideas are to use external 
and existing resources of information, rather than to record new clips for each subject. Another idea is to 
test how a RDF vocabulary can be associated automatically with an existing dictionary. 

The prototype CHESt was tested with a simple keyword search in some selected classes in a 
school in Luxembourg (Europe) during the summer term of the year 2004. The experience was very 
promising and prepared the field for a larger pilot project that will be launched in the winter term of the 
year 2004. This next prototype will work with a semantic search engine. We hope that the collected data 
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will help us to progress in the development of the retrieval mechanism and to prove, that education can be 
improved by the use of such "intelligent" e-learning tools. 
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Abstract 

Professional education is increasingly facing accessibility challenges with the emergence of web- 
based learning. This paper summarizes related U.S. legislation, standards, guidelines, and validation tools 
to make web-based learning accessible for all potential learners. We also present lessons learned during 
the implementation of web accessibility within an educational digital library tool called the Instructional 
Architect. 



Introduction 

E-learning is a rapidly growing trend that is providing increased educational opportunities for all 
learners. However, it also raises new concerns about how to accommodate students with disabilities. 
According to the United States Census Bureau (1997), 20 percent of the United States population has some 
kind of physical or cognitive disability, and one in ten has a severe disability. Moreover, 40 percent of 
college freshmen reported having some type of disability (Henderson, 2001). From an ethical perspective, 
these people deserve the same learning opportunities and ability to engage with web-based educational 
materials and resources. 

One key concern is web accessibility. The Web Accessibility Initiative (WAI) defines web 
accessibility as access to the web by everyone, regardless of disability, so that (a) people with disabilities 
can perceive, understand, navigate, and interact with web sites and web applications; (b) people with 
disabilities can use web browsers and media players effectively, which should work well with all assistive 
technologies that some people with disabilities use to access the web; (c) web authoring tools, and evolving 
web technologies should support production of accessible web content and web sites, and that can be used 
effectively by people with disabilities (Brewer & EOWG Participants, 2003). 

Designing accessible educational websites will make education available and meaningful to 
disabled people and help them communicate more effectively with the aid of information technology. 
Although the rapid development of assistive technology makes it more possible for individuals with a wide 
range of disabilities to gain access to computers and multi-media products (Camevale, 1999; Closing the 
Gap, 2001), much web-based content with inaccessible design remains unavailable to disabled students 
(Waddell, 1998). 

Web developers and instructional designers have the responsibility of making educational web 
sites accessible to everyone. Unfortunately many web developers and instructional designers just do not 
know how to make their sites universally accessible, or they choose to ignore the accessibility issue (Slatin 
& Rush, 2003). 

The goal of this paper is to clarify web accessibility issues and to offer resources and lessons to 
assist others in successfully developing accessible educational web sites. In this paper, we outline relevant 
United States legislation, standards, guidelines, and validation tools for web accessibility. We then present 
lessons learned during the implementation of web accessibility within an educational digital library tool 
called the Instructional Architect (IA). 

Relevant Legislation, Standards, Guidelines, and Validation tools 

Legislation 

Legislation in the United States that supports the development of accessible web sites includes the 
Rehabilitation Act of 1973 — specifically Section 504 and Section 508 (last amended in 1998), and the 
Americans with Disabilities Act of 1990 (ADA). The Acts are intended to protect the rights of disabled 
persons. According to Section 504 of the Rehabilitation Act, the federal statutes guarantee that disabled 
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learners will not be discriminated against because of their disability status, and all government agencies, 
fully or partially federally funded projects and schools will risk litigation if their web sites are not 
accessible to disabled students (Web Accessibility in Mind [WebAIM], 2003). As the ADA of cyberspace 
(Waddell, 1998), Section 508 of the Rehabilitation Act also provides the basis for the Electronic and 
Information Technology Accessibility Standards, which took effect in June 2001. Accordingly, accessible 
web design becomes a requirement for everyone — educators, web developers, and instructional designers 
— rather than an optional feature. 

Standards 

Standards from the World Wide Web Consortium (W3C) for HyperText Markup Language 
(HTML) 4.01, Cascading Style Sheets (CSS) levels 1 (CSS1) and level 2 (CSS2) specify syntax and 
functionality of tags and styles used in web sites. 

Some new structural elements and attributes has been added in HTML 4 to improve accessibility 
by enhancing highly structured documents, complementing non-textual contexts, and improving navigation. 
For example, the FIELDSET and LEGEND elements has been added to create form structure by organizing 
form controls into semantically related groups, and the new “title” attribute is provided to give a short 
description of any web element (for example, the “title” attribute can be used with the anchor <A> element 
to describe the nature of a link so that web users can decide whether to follow it.) 

CSS enables web authors and users to style HTML from document structure. For example, CSS 
allows web authors to achieve visual effects by using CSS spacing, alignment, and positioning properties. 
By using CSS, web authors can write simpler HTML and eliminate the use of meaningless images and non- 
breaking spaces ( ), which is non-standard mark-up, to create space around an element. Removing 
the CSS layout information in case browsers do not support them or users who deactivate them results in 
cleaner, more concise documents. Reference information and tutorials on standards -based web 
development can be found at http://www.w3schools.com/. Most recent browsers support most of CSS1 and 
much of CSS2. 

Guidelines 

Guidelines for web accessibility are coordinated and established by the Web Accessibility 
Initiative (WAI). The current version of the guidelines is the Web Content Accessibility Guidelines 
(WCAG) 1.0 (http://www.w3.org/TR/WCAG10/), which provides web developers with guidelines and a 
prioritized list of checkpoints for how to write HTML and other authoring languages to make web sites 
accessible. Among the prioritized list, Priority 1 checkpoints must definitely be adhered to. Priority 2 
checkpoints should be satisfied, and Priority 3 checkpoints may be addressed to achieve web accessibility. 
Currently WCAG 2.0 is still a W3C working draft. 

Working with the above web accessibility standards and guidelines will help designers develop 
predictable, reliable, and accessible web sites. 

Validation Tools 

Validation tools check web sites for compliance with web accessibility standards and guidelines, 
summarize potential accessibility concerns, and provide links to information about potential accessibility 
problems identified. Some examples of free online validation tools include: the W3C HTML Validation 
Service (http://validator.w3.org), Bobby (http://www.cast.org/bobby), Cynthia™ Says 
(http://www.contentquality.com/), and WAVE (http://www.wave.webaim.org/index.jsp). 

Lessons Learned During the Implementation of 
Web Accessibility within the I A 

The Instructional Architect (IA) (http://ia.usu.edu/) is an Internet portal designed for use with 
digital libraries of educational resources. It provides a tool for creating instructional materials using online 
resources (Recker, Dorward, & Nelson, 2004). To provide the same learning opportunities to all users, an 
accessible version of the IA was released at the end of 2003. The following summarizes lessons learned 
during the process of becoming aware of, obtaining training in, developing within, and testing against the 
standards and guidelines for accessible web sites. 

Awareness of Web Accessibility 
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Instructional designers and web developers cannot afford to not know or ignore accessibility issue, 
both from ethical perspectives and according to law mandates. Consider the form shown in Figure 1, which 
ignores web accessibility issues and is supposed to function as the form in Figure 2. 

Because the form fields and button in Figure 1 are not labeled following web accessibility 
guidelines, blind users who use assistive technology devices such as screen readers will have an experience 
similar to non-blind users accessing the form with the monitor turned off. 

Planning for accessibility in the outset avoids wasting resources during subsequent conversions of 
inaccessible applications. If instructional designers and web developers keep accessibility in mind and 
build an accessible site from the beginning, it becomes much simpler to develop and maintain the site cost- 
effectively. 

In addition, web developers and instructional designers should be aware that accessibility means 
accommodating a broad range of users, including visual impairmently users such as low vision, blind, and 
color-blind users, hearing impairmently such as deaf users, motor impairmently users, and cognitive 
disabled users. Cognitive disabilities remains the least understood and least discussed type of disability 
among web developers and instructional designers (Bohman, 2004). As a result, more research is needed to 
better understand and improve learning opportunities for such learners. 

Professional Training and Learning Community 

Currently many developers and designers are neither aware of the need for accessibility, nor know 
how to achieve it. Training programs with complete and well-organized web accessibility tutorials help 
with both of these issues. For example, the IA team benefited from comprehensive web accessibility 
training offered by WebAIM (http://www.webaim.org). 

Besides professional training, web developers and designers should participate in online web 
accessibility learning communities to update techniques and share experiences. Along with the evolution of 
web technologies, new accessibility issues will emerge. The IA team found it helpful to participate in the 
Accessify Forum (http://www.accessifyforum.com/ ). This provides a forum for sharing lessons and 
experiences among web accessibility experts, and for being informed about the latest technology and 
applicable resources. 

Development 

During the development of web accessibility for the IA, we experienced that to create an 
accessible web sites is not more difficult and time-consuming if it is properly p lanned for at the outset. In 
addition, we found that the following approaches very helpful in the development of the accessible IA. 

Use of Cascading Style Sheets ( CSS). The use of CSS enables accessibility primarily by separating 
document structure and content from layout and presentation. For example, CSS can help avoid "tag 
misuse". In our case, we initially used TABLE elements in every web page to align content. We later 
learned that disabled users would be confused and misled by this kind of tag misuse. One example of such 
confusion happened when blind users access web pages using a screen reader. Because a screen reader will 
read the table content in a linear way, complicated table structures used just for alignment purpose 
(embedded with rich COLSPAN and ROWSPAN attributes or nested TABLE elements) will disassociate 
the real data. As a result, it becomes impossible for blind users to understand the web pages effectively. 
Furthermore, because accessibility causes the HTML to be more completely defined, separating style from 
markup with CSS will cause web documents to be loaded more quickly and maintained more easily. In 
addition, web developers can verify that the simplified structure works effectively when CSS is not applied. 
Using the simplified structure, web developers only need to make changes to the CSS file to adjust the 
layout of the web site, rather than changing every page on the site. 

Creating an accessible web site doesn’t mean that the web site must be boring and graphics -free. 
Graphics increase accessibility by providing visual cues to help people to understand the content and 
navigation of the web site better. Following the CSS standards and web accessibility guidelines, the visible 
aspects of graphics will not pose inaccessible problems to blind users because web developers can provide 
equivalent alternative text (via the "alt" attribute) that allows blind users to access the content of graphics 
with their screen reader software. In short, web developers can build accessible web site with an attractive 
design by following web accessibility guidelines. For example, the CSS Zen Garden site 
(http://www.csszengarden.com) demonstrates how attractive web sites with different visual appeals based 
on the same web content can be built with the power of standardized HTML and CSS. In designing the 
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accessible IA site, we also benefited from using CSS by creating nine alternative PowerPoint-like 
background templates to provide useful feature for the IA users. With this enhanced feature, IA users have 
the opportunity to choose different fonts and backgrounds when creating instruction using the IA to match 
their subject and grade level. And as a result, their instruction is also accessible. 

There is one caveat we would like to share when using CSS during web accessibility 
implementation. CSS support differs between web browsers, even within different versions of the same 
web browser. Web developers should test the web site in as many browsers as possible and need to master 
commonly implemented CSS features to find the best CSS solution for the widest variety of browsers. 

Use of templates. The use of templates helped the IA web developers separate the HTML from the 
scripting code, which makes changing and maintaining both the code and HTML files much easier. This 
separation also prevents interface and web designers from breaking scripting code. Furthermore, accessible 
templates improved our efficiency in implementing accessibility in two ways: (1) by making the several 
general templates accessible, most of our work was vastly simplified; and (2) improvements to the 
accessibility of each page can be made by simply maintaining a few page-specific template files. 

There is no need to create a separate, text-only version. During the early stage of web 
accessibility implementation, there were some discussions about whether providing a separate text -only 
version would help meet the blind learners’ needs. In the end, we found it is unnecessary to maintain two 
versions concurrently. Not only it is hard to maintain two parallel versions, but we found that web 
developers can meet the needs of blind users by providing equivalent alternative text for all visual 
elements. In our approach, the web site has an inherent text version already because embedded alternative 
text will allow blind users to access the content of graphics and other visual materials. Here we want to 
point out again that it is a misunderstanding that web accessibility implementation means a "text only" 
approach for blind users. “Text only” practices may remove visual materials which blind users can not 
view, but also potentially reduce the accessibility of the site for the people with a cognitive disability as the 
visual materials will help sighted users, especially users with cognitive disabilities, to better understand the 
meaning and organization of the content. 

User Testing 

Validation tools are only computer programs, and cannot capture all potential problems with 
regard to the accessibility of a web site for disabled users. For example, W3C CSS validation tools can help 
check whether the correct syntax was followed, but they cannot check if CSS actually makes sense or if the 
resulting layout is actually what was intended. In the same way, Bobby or WAVE can only check whether 
the web site meet technical requirements; they cannot guarantee that disabled users’ experiences regarding 
web accessibility is equivalent or satisfying. 

As such, it is important to find disabled users to test the web site and make sure it is accessible for 
the widest possible range of disabilities — we recommend that this user testing occurs throughout the 
development process (Nielsen, 2000). 

The design of the accessible IA web site benefited from the suggestions of two blind users. 
Although we followed the WAI guidelines, these two blind users still encountered small accessibility 
barriers that did not trigger warnings from the validation tools. For example, the IA has a search function in 
which users type in keywords and are shown 20 hits at a time. For each hit on an online resource, the IA 
provides a link called “more info” showing its metadata. Although this page passed Bobby validation, one 
blind user was confused about the 20 “more info” links read by the screen reader. There was no way for 
him to figure out which “more info” link was associated with the specific online educational resource. With 
his feedback, we appended each resource’s title to each “more info” link. But in order to not impact on the 
original visual appearance of the page content, we implemented CSS to hide each appended resource’s title. 



Table 1 summarizes the key lessons the IA team learned during implementation of web 
accessibility, along with each lesson’s benefits and difficulties. 



Lessons Learned 


Benefits 


Difficulties 


Awareness of Issue 


From ethical perspectives and according to law 
mandates 

Avoid wasting resources during later conversions 


None 
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Participation in Professional 
Training 

and Learning Community 


Become knowledgeable web developer quickly 


Cost and Time 


Development: 
Use of CSS 


By separating document structure and content from 
layout and presentation, simplify HTML structure 
Web documents can be loaded more quickly and 
maintained more easily 


CSS support differs 
between web browsers, 
thoroughly testing is 
needed 


Development: 
Use of Template 


By separate the HTML from the scripting code, 
changing and maintaining both the code and HTML 
files are much easier 


Need to learn specific 
template syntax 


Development: 

No Separate Text Version 


No need to maintain two parallel versions 

The web site will have an inherent text version already 

if alternative text is provided for all visual elements 


Provide equivalent 
alternative text properly 
for all visual elements 


Iterative User Testing 


Only real disabled users’ experiences can ensure web 
accessibility, which no validation tool can guarantee 


It is difficult to find the 
widest range of 
disabled users 



Table 1. Lessons Learned in Implementing Web Accessibility. 



Discussion 

The IA development team made efforts to design a digital library tool accessible to all users, yet 
we are still facing web accessibility challenges when using online resources from digital libraries. We find 
that many of these online resources are inaccessible. For example, some video clips, audio files, images, or 
flash files don’t have equivalent text elements. Since online learning resources are the centerpieces of the 
IA, how do we continue to make the IA accessible if the resources that IA collects are not accessible? We 
realize that a good educational web environment can only be built with the cooperation of all instructional 
designers and web developers. It is the responsibility of instructional designers to initiate and raise 
awareness that all learning resources need to be created in accessible way. As such, instructional designers 
must become the change agents to disseminate web accessibility innovation. 

Furthermore, as the World Wide Web is global, web accessibility issues should be coordinated at 
an international level. We hope that countries all over the world consider web accessibility as an important 
endeavor, and that every country will commit to working cooperatively. 

Conclusion 

Web accessibility is not always easy to implement, often because many web developers and 
instructional designers simply do not know enough about the issue. Even some of the more informed 
developers and designers minimize its importance, or even ignore it altogether (Slatin & Rush, 2003). 
Because approximately one in five people has a disability affecting the quality of their web experience 
(Waddell & Thomason, 1998), implementing web accessibility must become a fundamental consideration 
in the design and development of web-based learning resources and instruction. 
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Figure 2 : Form with labeled input fields. 
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Abstract 

Contemporary learners encounter a diverse range of photographic materials. However, proximity, 
abundance, and familiarity do not insure that learners are prepared to exploit photographic images as an 
instructional resource. Visual literacy is cjuickly becoming a critical learning tool equal in importance to 
its text-based counterpart. This article examine how effective visual material use requires learners to 
develop an understanding of how photographs communicate ideas through their unique information 
potential, and how the context in which photographic information is presented affects its message. 

Introduction 

In their instructional role, visual materials are seldom viewed in isolation. The context in which 
the viewer encounters them can have a major influence on the information that the learners constructs from 
his/her viewing experience. Of all the diverse types of visual materials used in instruction photographic 
images may demand the most from the learners’ visual literacy skills. Photographic images are familiar 
elements in our daily lives. Because the content of a photographic image appears realistic and recognizable , 
at least within the culture in which it is created, the viewer tends to believe that he/she understands the 
photograph’s subject message without realizing how dependant that viewing experience is on 
accompanying contextual information that creates a framework which pre-loads the viewers’ expectations 
of the visual content. In an instructional setting, successfully reading photographic communication depends 
on learners possessing visual literacy skills equal to the text -based literacy skills taught in our classrooms. 
Yet, visual literacy instruction rarely receives significant attention from educators. 

The Neglected Literacy Partne r 

It is generally understood that learning to create meaning by reading patterns of text symbols, 
assembled as words, is a complex skill that takes time and effort to acquire. That the process of creating 
meaning from reading visual symbols also requires significant amounts of time and effort to acquire does 
not seem to be as obvious to many. Each medium of communication relies on its participants sharing a 
form of literacy. Whether it is a spoken language, a set of alphabet symbols, or other pictographic materials 
both the creator and receiver of the communication must share a mutual understanding of the meaning 
those symbols. This shared literacy is the Rosetta stone which facilitates the communication. Carroll 
believes that “each medium has certain inherent characteristics that determine what kinds of messages are 
transmitted by that medium and how those messages are attended to, understood, and acted upon overtly 
and covertly’’. Successful communication depends on both the message’s creator and viewer sharing a 
common literacy constructed to encode and interpret the inherent characteristics of the message’s particular 
medium. (Carroll, 1974) 

Photographic images present complex interpretative challenges because, in western society, 
viewers traditionally endow them with the power to present reality. The mechanical process of making a 
photographic image is the process of arresting time. The photographer selects a moment in time and space 
and freezes it through the technical process of making a photographic image. The viewer of a photographic 
image steps out of time and examines a micro-slice of the universe freed from the twin tyrannies of time 
and motion. This affords the viewer a unique perspective from which to observe and discern information 
that would normally flash by at the speed of light. Any single photographic image is only a micro -slice of 
time and space. The photographer determines the contents of each micro-slice and, by the process of 
making the photographic image, excludes all the space and time that surrounds each image. By its nature, 
this process limits the information available to the viewer. The choice of the elements included in any 
photographic image does not belong to the viewer. The viewer must understand both the mechanics of the 
photographic process and the selections and limitations that result from that process. Further, by its nature 
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